Context

Novo Nordisk has numerous warehouses and production facilities where humans, manually operated forklifts, and mobile robots share space. To ensure an optimal operation of these heterogeneous agents, different strategies need to be put into place, including human-free zones and prioritization depending on criticality of each agent’s mission (e.g. in specific cases manual forklifts might be prioritized over mobile robots due to size and load). Additionally, to avoid traffic jams, we need to ensure that areas with high traffic are free from objects that are not meant to be there, such as boxes and pallets.

Challenge 

We want to leverage drone technology for warehouse safety by deploying autonomous drones to ensure human-free zones and detect obstacles within warehouse aisles. Using a simulation platform, we want to replicate real-world warehouse scenarios, enabling the drones to navigate the space, identify human presence in designated no-human zones, and detect obstructive elements such as boxes on the floor. The drones should autonomously navigate around the space, capture and record images of these occurrences, and describe the issue with VLMs, enhancing warehouse safety and operational efficiency.

Tools, methods and materials

• Scene of a warehouse in simulation, with some typical objects to be detected.
• Robot simulation.

Ideal outcome for the company 

Code to control a drone in a simulation, collect images, process them with a detection algorithm and provide the scene description with vision language model. This will used as a proof-of-concept before building a similar system in real-world scenarios.