Catch It! Learning to Catch in Flight with Mobile Dexterous Hands

Abstract

Catching objects in flight (i.e., thrown objects) is a common daily skill for humans, yet it presents a significant challenge for robots. This task requires a robot with agile and accurate motion, a large spatial workspace, and the ability to interact with diverse objects. In this paper, we build a mobile manipulator composed of a mobile base, a 6-DoF arm, and a 12-DoF dexterous hand to tackle such a challenging task. We propose a two-stage reinforcement learning framework to efficiently train a whole-body-control catching policy for this high-DoF system in simulation. The throwing configurations are randomized during the training to improve policy adaptivity for different trajectories of objects in flight. The results show that our trained policy catches randomly thrown trajectories of diverse objects at a high success rate of about 80% in simulation, with a significant improvement over the baselines. The policy trained in simulation can be directly deployed in the real world with onboard sensing and computation, which achieves catching flying objects thrown by humans.

Catch Diverse Objects in Real-World

Method

Sim2Real Illustration of Catching Motions

Catch It! is a method using reinforcement learning (RL) to learn a whole-body control policy for catching objects in simulation, with Sim2Real transfer to real robots.

Two-Stage Training Framework in Simulator (Better Efficiency)

Stage 1: Training for Tracking

The beginning of training

We first train the base and arm control policies in a tracking task parallelly. During the initial phase of training, it is evident that the robot is unable to follow the object's trajectory.

After training for a while

After a period of training, the robot has learned to track the trajectory of the object and make contact with it using the palm.

Stage 2: Training for Catching

The beginning of training

We continue to train the robot's catching policy. At the early stage of training, the robot tracks the object easily but fails to grasp it.

After training for catching

Through the two-stage training approach, the robot ultimately achieves the ability to track and catch objects.

Generalize to unseen objects

To validate the policy's generalization in simulation, we test with 5 unseen object shapes. And our trained policy demonstrates robust generalization to unseen objects.

Failure Cases

When deploying the trained policy in the real world, our system may struggle with elastic objects, which tend to rebound off contact surfaces, particularly when both of the object and robot move at high speeds.

A Funny Case

In this intriguing fishing game, Catch It! demonstrates the capability of adaptive long-horizon tracking.

BibTeX

@article{zhang2024catchitlearningcatch,
  title={Catch It! Learning to Catch in Flight with Mobile Dexterous Hands},
  author={Zhang, Yuanhang and Liang, Tianhai and Chen, Zhenyang and Ze, Yanjie and Xu, Huazhe},
  journal={arXiv preprint arXiv:2409.10319},
  year={2024}
}