The overall objective of First-MM is to develop the technology for a new generation of autonomous mobile manipulation robots which can be instructed easily to execute mobile manipulation tasks in a robust way. This will be achieved by developing a framework that allows users and developers to easily specify programs for mobile manipulation activities. Among standard instructions for navigation and manipulation, this programming environment will include abstract skills for robust navigation and manipulation actions. These abstract actions are concretized by the robot through perception, imitation learning, interaction, relational learning, and robust navigation. More precisely, the robot will be able to deal with uncertain perceptions and to learn manipulation skills through a continuum of interaction between learning from experience and learning from interacting with humans. This will be realized by extending and integrating advanced techniques from mobile robotics, perception, machine learning, interaction, and learning from demonstration. The results of First-MM will help to close the gap between special-purpose robots that robustly execute manipulation tasks in the real world and advanced approaches to robot navigation, statistical learning, task specification, and probabilistic reasoning.

To evaluate the scientific achievements of the First-MM project, state-of-the-art mobile platforms will be extended with a novel software architecture for mobile manipulation tasks. This integrated system will serve as a prototype that is able to flexibly solve mobile manipulation tasks like object transportation activities in complex and dynamic environments.