How do you teach a robot to see not just a stack of boxes, but the thin slip sheet between the layers and the stretch band holding them together? 

For the expert team at EEP-Robotics GmbH, their answer to this complex challenge didn’t just solve a customer’s problem. It won them the 1st prize at our Photoneo Technology & Applications Conference 2024.

By building their solution around four of our PhoXi 3D Scanners L, EEP engineered a truly intelligent and autonomous order picking system.

The Challenge: Automating Complex, Unstructured Order Picking System

EEP’s client needed to automate the depalletizing of a wide variety of cardboard boxes and plastic load carriers (KLTs) from Euro pallets.

The market’s increasing demand for automation requires solutions that go far beyond established methods, with particular demand on flexibility, speed and intelligence.

The primary challenges were:

• Unstructured Pallets: Boxes and carriers were not arranged in a defined pattern, requiring a vision system to identify each item’s precise location and orientation, including the joints between boxes.
• Object Variety: The system had to handle 7 types of load carriers and cardboard boxes with dimensions ranging from 128 x 62 x 65 mm to 395 x 265 x 270 mm.
• Complex Layers: Pallets included intermediate cardboard layers, wooden attachment frames, and stretch bands on each layer, all of which needed to be detected and handled by the robot.
• Large Scan Volume: The full height of the pallets exceeded the scanning range of a single static scanner, posing a significant challenge to achieving high-resolution scans from top to bottom.
• High Performance: The client required a minimum throughput of 250 picks per hour, including time for 10 robotic gripper changes.

EEP recognized that without a top-tier 3D vision system, meeting these demands for speed, accuracy, and reliability would be impossible.

The Solution: PhoXi 3D Scanners with a Hybrid Software Approach

EEP selected Photoneo after a thorough evaluation of multiple 3D vision providers, ultimately deciding to go with PhoXi 3D Scanner devices, noting that it was important to“have a strategically good partner with a lot of know-how and experience in 3D-vision applications.”

Four PhoXi 3D Scanner L units were mounted on a ceiling servo axis system. This innovative mechanical setup allows the scanners to move vertically, effectively extending their scanning range to cover the entire pallet height while maintaining the high resolution needed for precise position detection.

The core of the solution lies in a unique, hybrid software architecture that combines the strengths of Photoneo’s native software with EEP’s custom AI algorithms running directly on our scanner’s controller.

• Photoneo’s Onboard Software: Our powerful, built-in object recognition software was used for its “plug & play” capability to reliably detect and locate all cardboard boxes and plastic carriers on the top layer.
• EEP’s Custom Docker Application: A key decision factor for EEP was the ability to run their own custom software in a Docker container on the Photoneo controller. This EEP-developed software 4 Phoxi 3D Scanner units mounted on servo axisprocesses the raw point cloud to perform specialized tasks, including the detection of intermediate cardboard layers, wooden pallet frames, and stretch bands.

This data flow is best illustrated by EEP’s communication architecture:

This infographic shows a step-by-step breakdown of the process:

1. Trigger from PLC: The process begins when the main robotic cell controller (EEP PLC) sends a “trigger” command to start the vision task.
2. EEP’s Software Receives Command: The command goes to EEP’s custom software environment, which acts as the central brain. This software parses the trigger to understand what the PLC is asking for.
3. Standard Photoneo Scan: EEP’s software then triggers the standard Photoneo application. The scanner performs a scan and uses its built-in pipeline to quickly detect standard objects like boxes. These results are immediately sent back to EEP’s software.
4. Simultaneous Raw Data Access: In parallel, a custom function allows EEP’s software to access the raw 3D point cloud data generated by the scan.
5. Custom AI Processing: EEP’s software takes this raw data and runs its own advanced algorithms (using technologies like TensorFlow and PyTorch) to perform specialized detections for items like intermediate layers and stretch bands.
6. Combine and Process Results: Finally, EEP’s software combines the standard results from Photoneo’s pipeline with the custom results from its own analysis. It processes this unified data into a single, comprehensive command set and sends it back to the EEP PLC.

In short, EEP brilliantly uses Photoneo’s fast and reliable onboard software for standard detection while simultaneously leveraging the system’s openness to run their own advanced AI for custom tasks, creating a highly flexible and intelligent vision solution.

A Step by Step Workflow of Order Picking System

The floor operates a single robot that intelligently alternates between two parallel floor “spheres”: one for handling boxes and the other for handling KLTs (Small Load Carriers). The robot works in perfect synchronization with Photoneo 3D scanners and autonomously changes grippers to cover all the jobs entailed in this complex application.

Here is the step-by-step explanation of the entire cycle:

1. Automatic delivery of the pallet in the robot cell: The process begins with the pallet being automatically delivered and positioned inside the robot’s working cell, ready for processing.
2. Pallet scanning with PhoXi 3D scanner on servo axis: A PhoXi 3D scanner mounted on a servo axis performs a detailed scan of the pallet and its contents. This scan provides the robot with the exact location, orientation, and dimensions of the items to be picked. 
3. Grab the right gripper with the Kuka robot: Based on the scanning results and the task (handling a box or a KLT/tray), the Kuka robot autonomously selects and equips the appropriate gripper from its tool changer.
4. Pick the box or tray: The robot uses the equipped gripper to pick the specified box or tray from the pallet.
5. Labelling of the box or tray: The picked item undergoes a labeling process to identify its contents and destination.
6. Reading the RFID Tag of the tray: If the item is a tray, its RFID tag is read to confirm its identity and track its progress through the system.
7. Place the product in the tote provided on the customer conveyor system: The final product is carefully placed into a tote or container that is located on the customer’s conveyor system for further transport or processing.
8. Automatic delivery of the pallet in the robot cell: After the product is placed in the tote, the cycle completes, bringing us back to the start – ready for the next pallet delivery or continuing with the current pallet until all items are processed.

Results and Benefits: A Prize-Winning Advantage

The project, from initial tests to go-live, took approximately one year due to its complexity. The result is a robust, well-trained 3D vision system that has delivered significant benefits.

• High Performance and Accuracy: The system successfully meets the performance target of 250 picks per hour and accurately handles the diverse range of products and pallet complications. According to EEP, the AI behind Photoneo’s technology works “very, very well.”
• Ease of Use & Powerful Customization: EEP praised the user-friendly nature of our products, noting, “You get good results very quickly without much effort.” They specifically highlighted that the “cardboard box detection works as a plug & play solution quickly and reliably.” This, combined with the ability to deploy complex custom algorithms, provided the perfect balance of simplicity and power.
• Reusable and Scalable Solution: EEP has developed a well-trained vision system that they can now deploy in other applications without new development, creating a significant return on investment and a technological advantage in the market.
• Strategic Partnership: EEP valued having an experienced European supplier with a large development team. The seamless integration and support throughout the process confirmed their choice of Photoneo as a strategic partner.

As Ing. Horst Hörmann of EEP-Robotics stated, the single biggest reason to recommend Photoneo is that “There is a suitable scanner size for every application and the AI behind it works very, very well.” 

This award-winning application is a clear example of how integrators can use Photoneo’s advanced 3D vision to build the next generation of automation solutions.

 Dowanload the PhoXi 3D Scanner Brochure to learn more: