2. October 2018
Machine Learning is essential to the development of artificial intelligence and in the emergence of autonomous systems and technologies. This is exactly what makes our robots stand out: they act autonomously, even in dynamic, unknown environments. To do this successfully, they need to constantly learn by drawing conclusions from a set of data and adapting their actions accordingly.
Of course, all our robots have excellent manners: we provide them with a lot of useful information, even before they started working. This includes, for example, basic action and movement sequences for gripping and setting down safely, correct speeds in various situations, and default reactions to obstacles and errors. But the warehouse environment is complex and constantly changing. With people and objects moving in the environment, the robot cannot successfully react to changes solely with its initial knowledge because these movement patterns are complex. An example: the robot wants to pick a box from the shelf rack, but it has not been correctly placed by a human colleague. The robot now faces a major challenge: where exactly should it reach? This is where Machine Learning comes into play.
The different approaches of Machine Learning
We can train the robot to solve problems in different ways: on the one hand, similar to human learning, there is “supervised learning”. Here, the system gets a lot of examples and images for comparison, as well as the corresponding results as input. However, since our robots are themselves able to perceive, constantly collecting their own data through cameras and sensors, they can also create a “supervised learning problem” for themselves – we at Magazino have called this type of Machine Learning “self-supervised learning”. Using intelligent algorithms, the robot can abstract empirical values or data and draw conclusions from them: if, for example, the robot compares the planned with the observed effect of its action, or the world before and after its action, it can learn a model of its action. This can be used to predict which effects can be expected given certain initial conditions and action parameters, such as during object segmentation or grasping. This means a model of the object as a delta between before and after, and a method that can also distinguish other objects from sensor data.
A third type of Machine learning is “reinforcement learning”, which combines both of the previous variants. In addition, incentives are given to the system via a reward or penalty system. A potential area of application at Magazino is order distribution between robots, i.e. within a robot fleet. Penalty points are given for each passing meter or second, and reward points are awarded for the successful picking or putting down of objects.
But Machine Learning is complex, especially for mobile robots with a variety of sensors.
One of the challenges here is that the sensor data obtained are extremely different – not comparable to two-dimensional digital images that are processed by simple image recognition software. The robots must be able to filter meaningful image sections: this means that they must be able to decide which data from the cameras is actually relevant to them at that moment.
You can imagine it like this: when people take photographs, they usually focus on a certain object, which is then relatively prominent in the picture. If a robot captures images or perceives, this happens initially without a special focus.
Therefore, the next step is to find out where an object begins, ends and how far away it is from the camera or sensor. This becomes relevant, for example, when the robots are standing in front of a tightly packed shelf full of shoe boxes. They then have to identify the correct boxes among them and plan the corresponding gripping movement precisely, so that the other boxes are not disturbed.
Where is Machine Learning particularly useful
Our robots learn through feedback cycles, which we call “self supervised learning”: perception, reaction, feedback, insight. This is very clear in the application case “identifying optimal gripping points on shoe boxes”. Here, the gripping motions are optimized for the identified gripping points:
This is how learning works
- With our mobile picking robot TORU, data from numerous “picks” of shoe boxes are recorded during live operation.
- A special focus is on which of the six suction cups of the vacuum gripper form seals with the box during picks and can therefore generate a sufficient suction force.
- At the same time, before each pick, a photo is taken with the camera in the gripper and linked to the respective results of the subsequent pick.
- Even if almost all picks are successful, about half of the individual suction cup feedback is negative.
- The data obtained is fed into a neural network. Based on the output, a heat map is created to show in which areas of the box the suction cups tend to be successful, and where they tend not to be.
- The system is then shown the image of a completely new shoe box. On the basis of the previously gained knowledge, the system can identify the areas on the box where the greatest likelihood for the six suction cups to successfully create a vacuum are. The robot has thus learned successfully and can continue to apply its knowledge in the future.
The effective use of Machine Learning constantly improves the performance of our robots and makes them adaptable. With their intelligent algorithms, they can easily react to novel situations, such as successfully picking new or shifted boxes, or finding new ways through the warehouse. And since our robots are networked locally and globally via a cloud, a robot learns not just for itself, but for the entire fleet: if a TORU in the warehouse gains a new insight into a blocked passage, it shares this with all others in real time. In the future, the deployment process will also be significantly shortened, converging to full automation. Our long-term goal: after initial setup, the robots can quickly and efficiently find their way around their new environment, immediately providing active support and added value from day one.