Taking Mobile Manipulators into the Real World

Jesse Haviland

Introduction

Fabio Ramos

Differentiable Physics Simulators for Manipulation: Challenges and Opportunities

The availability of differentiable programming languages enables the development of differentiable simulators capable of providing derivatives with respect to physics parameters and dynamics states. This opens numerous opportunities to connect modern probabilistic inference with the simulation process to infer physical parameters from real data, or to learn policies more effectively. In this talk I will present examples of methods that leverage differentiable simulators for manipulation of deformable objects, robot cutting, locomotion, and learning the structure of articulated objects. I will show that when combined with probabilistic inference, real2sim (adapting the simulator to match real data) can be done robustly with just a few observations.

Georgia, Fabio, Firas, Lukas, Rika

Panel/Round Table. Chaired by Tirtha

Identify the open challenges in real-world and field: manipulation, interaction and mobile manipulation. This will identify the technical, scientific and application related bottlenecks to deployment of robots to real-world environments.

Poster Session and Break

Georgia Chalvatzaki

Structuring Robot Learning for Mobile Manipulation

The increasing demand for intelligent robotic assistants in unstructured and human-inhabited environments, such as homes, hospitals, and warehouses, necessitates the development of more efficient, scalable, and safe robot learning methods. In this talk, I will discuss our research at the intersection of classical robotics and machine learning, focusing on structured learning approaches that enable mobile manipulation robots to better understand and interact with their environments. By exploiting the structure of the problem or by imposing structure as inductive bias, we enhance the learning process for real-world applications of mobile manipulation.

Danica Kragic

Learning Interaction for Robotics Tasks

Leonardo Bobadilla

Spotlight Talk From Interaction to Integration: Advancing Optimal Human-Robot Interfaces for Underwater Manipulation

Liangjun Zhang

Spotlight Talk Autonomous Excavator System for Construction Automation

Sophie Lueth

Spotlight Talk Looking Good: Visually Informative Motion Generation for Mobile Manipulation

Lukas Kaul

Mobile Manipulation @TRI: From the (corporate) Lab into the Real World

Challenge tasks (akin to the DARPA Robotics Challenge) can effectively drive fundamental research, while real-world testing provides an invaluable feedback mechanism to ensure research efforts translate into meaningful results. This talk will introduce TRI's approach to mobile manipulation research and reflect on the state and future of mobile manipulation research.

Firas Abi-Farraj

Walking the Tightrope: Simplifying the World vs. Complexifying the Robots

While mobile robots are (slowly) finding their way out of the lab, mobile manipulators are lagging behind. In this talk, I'll briefly discuss the reasons why and focus on our recent efforts to re-shape the mobile manipulation problem in a way that allows for deploying mobile manipulators in social environments (as soon as 2025!).

Luis Sentis

Control and Intelligent Legged Manipulation in Unstructured Natural Environments

Humanoid robots remain one of the most complex robotic systems to control and perform useful tasks. There are two forces driving the need for humanoid robots. On one hand blue collar jobs such as cleaning, inspecting, and moving objects around. On the other hand, with the rise of large language models there are chances that human-centered robots could play a role in the knowledge economy. In any case, performing relatively simple dexterous manipulation tasks in floating based robots (e.g. humanoids) remains a difficult challenge. Another dichotomy is the duality between optimal control methods and deep learning methods for control of human-centered robots. In this talk we focus on control and embodiment by exploring the combination of model based and model free approaches. Optimal control methods are used to track trajectories of humanoid robots with high fidelity. At the same time, we employ imitation learning techniques to accomplish two capabilities: social locomotion in crowded environments and dual arm manipulation of complex objects using humanoid robots.