This workshop seeks to progress research in the topic of real-world manipulation and mobile manipulation, that is manipulators operating in environments which may be outdoors, naturally occurring, and unstructured. These real-world environments have unique characteristics making them non-trivial and difficult for robot operation compared to typical laboratory environments. Challenges include travelling over uneven and unpredictable terrain, working with deformable and non-rigid structures such as trees and other natural phenomena, and operating with high uncertainty due to environmental conditions.
This workshop aims to bring together researchers from diverse sub-communities within robotics including manipulation, mobile manipulation, field robotics, legged robotics, robot learning, SLAM, and robotic vision to discuss the following topics:
From Interaction to Integration: Advancing Optimal Human-Robot Interfaces for Underwater Manipulation
Autonomous Excavator System for Construction Automation
Looking Good: Visually Informative Motion Generation for Mobile Manipulation
IndoorSim-to-OutdoorReal: Learning to Navigate Outdoors without any Outdoor Experience
Reinforcement Learning Based Escape Route Generation in Low Visibility Environments
Royal Institute of Technology
University of Texas
Toyota Research Institute
NVIDIA, University of Sydney
|1:30 - 1:35||
|1:35 - 2:00||
Differentiable Physics Simulators for Manipulation: Challenges and Opportunities
The availability of differentiable programming languages enables the development of differentiable simulators capable of providing derivatives with respect to physics parameters and dynamics states. This opens numerous opportunities to connect modern probabilistic inference with the simulation process to infer physical parameters from real data, or to learn policies more effectively. In this talk I will present examples of methods that leverage differentiable simulators for manipulation of deformable objects, robot cutting, locomotion, and learning the structure of articulated objects. I will show that when combined with probabilistic inference, real2sim (adapting the simulator to match real data) can be done robustly with just a few observations.
|2:00 - 3:00||
Georgia, Fabio, Firas, Lukas, Rika
Panel/Round Table. Chaired by Tirtha
Identify the open challenges in real-world and field: manipulation, interaction and mobile manipulation. This will identify the technical, scientific and application related bottlenecks to deployment of robots to real-world environments.
|3:00 - 3:30||
Poster Session and Break
|3:30 - 3:50||
Structuring Robot Learning for Mobile Manipulation
The increasing demand for intelligent robotic assistants in unstructured and human-inhabited environments, such as homes, hospitals, and warehouses, necessitates the development of more efficient, scalable, and safe robot learning methods. In this talk, I will discuss our research at the intersection of classical robotics and machine learning, focusing on structured learning approaches that enable mobile manipulation robots to better understand and interact with their environments. By exploiting the structure of the problem or by imposing structure as inductive bias, we enhance the learning process for real-world applications of mobile manipulation.
|3:50 - 4:10||
Learning Interaction for Robotics Tasks
|4:10 - 4:20||
Spotlight Talk From Interaction to Integration: Advancing Optimal Human-Robot Interfaces for Underwater Manipulation
|4:20 - 4:30||
Spotlight Talk Autonomous Excavator System for Construction Automation
|4:30 - 4:40||
Spotlight Talk Looking Good: Visually Informative Motion Generation for Mobile Manipulation
|4:40 - 5:00||
Mobile Manipulation @TRI: From the (corporate) Lab into the Real World
Challenge tasks (akin to the DARPA Robotics Challenge) can effectively drive fundamental research, while real-world testing provides an invaluable feedback mechanism to ensure research efforts translate into meaningful results. This talk will introduce TRI's approach to mobile manipulation research and reflect on the state and future of mobile manipulation research.
|5:00 - 5:20||
Walking the Tightrope: Simplifying the World vs. Complexifying the Robots
While mobile robots are (slowly) finding their way out of the lab, mobile manipulators are lagging behind. In this talk, I'll briefly discuss the reasons why and focus on our recent efforts to re-shape the mobile manipulation problem in a way that allows for deploying mobile manipulators in social environments (as soon as 2025!).
|5:20 - 5:40||
Control and Intelligent Legged Manipulation in Unstructured Natural Environments
Humanoid robots remain one of the most complex robotic systems to control and perform useful tasks. There are two forces driving the need for humanoid robots. On one hand blue collar jobs such as cleaning, inspecting, and moving objects around. On the other hand, with the rise of large language models there are chances that human-centered robots could play a role in the knowledge economy. In any case, performing relatively simple dexterous manipulation tasks in floating based robots (e.g. humanoids) remains a difficult challenge. Another dichotomy is the duality between optimal control methods and deep learning methods for control of human-centered robots. In this talk we focus on control and embodiment by exploring the combination of model based and model free approaches. Optimal control methods are used to track trajectories of humanoid robots with high fidelity. At the same time, we employ imitation learning techniques to accomplish two capabilities: social locomotion in crowded environments and dual arm manipulation of complex objects using humanoid robots.