How depalletization can benefit from computer vision, robotics, and machine learning

How depalletization can benefit from computer vision, robotics, and machine learning

By Andrea Ferkova || June 1, 2020

Thanks to advances in technologies such as 3D machine vision, robotics is entering all spheres of industrial processes. The COVID-19 outbreak has boosted awareness of the pressing need to apply automation for fast and accurate order fulfillment and efficient supply chains. One specific aspect of logistics processes can achieve higher levels of productivity with the implementation of smart automation — depalletization.

What is depalletization?

Depalletization is the process of unloading pallets laden with boxes one by one. In contrast to the classic delayerization, in which the robot gripper picks the whole pallet, “hoping” it grabbed all the boxes it contains, robotic depalletization uses artificial intelligence. The system recognizes individual boxes and the robot places them one by one on a conveyor belt or other predefined place.

Depalletization presents a higher level of unloading pallets and offers several advantages over delayerization. It requires a smaller placement area — the size of the largest box in contrast to the whole pallet. And thanks to a lighter payload, a smaller robot arm and gripper can be used, which leads to considerable cost savings.

3D vision and AI the secrets to success

The best depalletization systems can make the process of unloading pallets full of various boxes safe, fast, effective, reliable, and in the end also cost-efficient. This can only be achieved by combining 3D machine vision with smart robots enabled by advanced machine learning algorithms.

The scanning volume of the deployed 3D scanner needs to be large enough to scan the whole pallet from sufficient distance. Taking into consideration the minimum space required for robotic manipulation, the scanner generally needs to be mounted approximately 3 meters above the pallet. Choosing the right 3D scanner is, therefore, the first prerequisite for successful depalletization.

The second step is training AI with this image data. Machine learning algorithms can constantly learn and recognize new types of boxes, including those of different sizes or irregular shapes caused by damage, for instance. This makes the solutions so universal that unloading of mixed pallets poses little challenge. The boxes do not need to be stacked in patterns but can be placed randomly, even tilted at an angle, and the robot is still able to pick them.

Smart systems are also able to recognize boxes that are often so tightly packed that it is difficult to recognize the gap between them, which can be as thin as 0,5 millimeters. Weaker solutions might not be able to differentiate the line separating two boxes from a line contouring the opening of one particular box.

In other cases, it may be challenging to recognize boxes with problematic surfaces, including varying textures, shiny or reflecting material, protruding tapes, patterns, or pictures that “mislead” the 3D vision. Cartons with black covering can also cause problems..

The most advanced way to segment the individual boxes on the basis of texture and 3D data is to use a convolutional neural network (CNN). The system can then decide which box to pick — boxes placed on the very top of the pallet come first — and how to grab it to maximize the suction power of the deployed gripper.

To manipulate safely in the space between the top boxes and the scanner, the depalletization system needs to take into account the possible size of the box. This is also important for safe placement of the box on a conveyor belt. That can either be ensured by calculating the height of the box from the scan data or by using an optical gate set to a few centimeters above the conveyor belt.

When the box touches the optical beam, the gripper drops it. This way, all boxes get dropped off in the same height above the conveyor belt. This is a big advantage of depalletization over delayerization, in which boxes of different heights in one layer pose a significant problem.

The robot can do the whole job

Photoneo’s systems, which include 3D vision developed in house, provide an example of such robotic depalletization. The company’s system scans an entire pallet loaded with boxes and transfers the scan to a 3D-texture data set.

This scan is then processed by Photoneo’s machine learning algorithm trained on more than 5,000 types of boxes. AI immediately recognizes each box and sends a command to the robot. Using a specially developed universal gripper, the robot performs the picking action with an accuracy of +-3 mm. This way, it is able to unload 1,000 boxes in our hour, with 99.7% pick-rate accuracy.

Depalletization systems must account for variability

If, despite all calculations, the gripper fails to pick a box due to a crinkled surface or some other obstacle, the gripper sends feedback and informs the user about the problem so that corrective action can be performed. The cycle time is typically less than 10 seconds, depending on the robot type, the surface of the boxes, and their contents, as some need to be manipulated with greater sensitivity than others.

The environment, robot, and mechanical properties define and limit the cycle time. For example, it would be impossible to accelerate and decelerate a heavy box above a certain physical limit. In case customers need to speed up the cycle time and boost the robot performance, they can opt for a multi-zone gripper that is able to pick several boxes of the same height at a time. The gripper then drops the boxes one after another.

The secret behind perfect singulation is to know the size limits of the placement area so as not to take more boxes than can safely be dropped off, and also to precisely recognize the box type to avoid grabbing boxes with different heights. Photoneo‘s product is compatible with major robot brands and works “out of the box” without any training. If it comes across new types of boxes, the system is able to retrain itself, which shortens the time needed for deployment and integration.

Adapting depalletization to a human-centric environment

A successful depalletization solution must take into account all the factors discussed above. Even though the robots may seem rather simple, the machine learning algorithms need to be robust enough to handle all the possible challenges of depalletizing different objects. Developers and integrators have to think about every detail of the application and test their solutions before users can measure return on investment (ROI).

In addition, it is often necessary to adapt the robot to a human-centric environment. Although automation is evolving quickly, many customers are only gradually adapting their distribution centers and warehouses to take full advantage of robotics and AI.

One of the major challenges related to manual unloading of pallets resides in the size and weight of the boxes as well as the height from which they need to be taken. Manual operations often lead to serious injuries, so the best depalletization systems help associates avoid risky or repetitive motions.

For instance, the Photoneo Depalletizer can pick boxes of up to 50 kg without human intervention. One of the greatest advantages is that the robot can work non-stop, without ever getting tired.

Depalletizer using 3D vision, AI, and a robotic arm

AI = unlimited potential?

AI-driven solutions are undoubtedly the way to the future because users do not need to design, debug, or test anything. Smart systems can relieve integrators of the burdens of difficult 3D-related calculations and tasks.

What they should have, however, is some basic mechanical knowledge, including how the different types of grippers work, which ones are suitable for picking a particular part, and how to distribute all mechanical components deployed in an application to prevent failures of the robotic manipulation or scanning.

Integrators should also know the potential capacities of a particular system to be able to match it with the specific needs of a customer. It is important to bear in mind that AI is still only a part of the solution and should not be overestimated. Integrators will always need certain specific knowledge to successfully deploy a smart automation solution.

If all these conditions are met, the deployment of depalletization robots can help logistics companies obtain a fast ROI, improve their supply chain processes, and increase their productivity. They can also save time, reduce costs, and protect worker health, freeing employees for tasks that require creativity and critical thinking.

The most pressing challenge facing both employees and employers today is how to keep working. Because no one can predict with any confidence when the global economy will recover from COVID-19 shutdowns, now is the right time to automate and streamline production processes.

This article was originally published in The Robot Report.

About the Author

Andrea Ferkova
Andrea Ferkova
Sr. PR Specialist || Website

Andrea Ferkova is senior public relations specialist at Photoneo and writer of technological articles on smart automation powered by robotic vision and intelligence. She has a master’s from the University of Vienna.

Recent blog posts