Context

At the LEGO group, our bricks are primarily created through the process of injection moulding. This ensures a precise and clean product after just one process. However, some of our elements require a post processing procedure, like printing, to ensure the highest level of quality and detail. To enable the printing process, it is crucial to constrain the elements in custom jigs, to guarantee that the prints are correctly aligned and printed. This process of constraining the elements is very challenging as almost all elements are produced and stored in large disorderly quantities in containers.

A large part of handling our elements is identifying them.

Identification of elements consists of using machine vision to detect correctly aligned single elements which are ready to be picked up by a robot. For the sake of simplicity.

These 2 areas have a close relationship, as one influences the other. As an example: it is a requirement of the segmentation solution to be able to isolate element from a cluster, if the Identification solution is not able to detect elements clustered together.

Challenge 

Develop a computer vision solution, to identify and estimate the pose of certain LEGO bricks, for a given vibration table equipped with a camera, lights, and vibration mechanism.

The computer vision solution should be able to identify and estimate the pose of elements located on a vibration table, based on a 2D RGB image. The vibration table will be at a standstill for image acquisition. The elements are listed in table 1 and are of varying materials. There is no strict execution time constraint for the solution, but execution time is a performance metric.

Tools, methods and materials

• Documentation of all the equipment
• Sample Python Code for camera and light control
• The physical test platform

Ideal outcome for the company

New ways to identify elements and their pose using 2D images.