How do we teach a robot to pick up an object?
The traditional approach is to program the robot step by step.
But as the number of tasks increases and environments become more complex, it becomes increasingly difficult to define every possible situation in advance.
Another approach is to:
Let robots learn from how humans perform tasks.
This is also the goal of the Human Demonstration Data service developed by Saltlux Technology for Physical AI and Robotics applications.
Capturing How Humans Perform a Task
During the data collection process, participants are equipped with data recording systems and asked to perform predefined tasks.
Instead of simply recording a person picking up a product from a shelf, the data is designed to capture the entire sequence of actions:
Observe → Approach → Grasp → Move → Place
This is known as a Human Demonstration – essentially, a real-world example of how a person completes a task.
Controlled Data Collection in the Lab
The lab plays an important role in this process.
Within a controlled environment, the same task can be repeated multiple times while selected variables are deliberately changed.
For example:
- placing objects in different positions;
- using objects of different sizes;
- varying the grasping method;
- changing the order of actions;
- or performing a task using both hands.
This allows the team to control key variables and build datasets tailored to the requirements of each specific use case.
From the Lab to Real-World Scenarios
The next step is to move data collection into real-world environments.
In retail stores, participants may need to work among a large number of different products.
In factories, tasks may involve materials, tools, and machinery.
Some tasks also require both hands to work together. In robotics, this is commonly referred to as bimanual manipulation.
This type of data can be especially valuable for research into robots that are expected to perform increasingly human-like tasks.
Data for Robot Learning
After collection and processing, Human Demonstration Data can serve as a valuable data source for research areas such as:
Imitation Learning – enabling robots to learn by imitating examples;
Robot Learning – helping robots learn how to perform tasks from data;
Embodied/Physical AI – developing AI systems that can perceive and act in the physical world.
Saltlux Technology’s service focuses on a critical part of this process: turning real-world human actions into data that AI systems can actually use.
