Research Areas

We aim to develop fundamental learning methodologies for whole-body manipulation using humanoid robots, including humanoid robots (Unitree G1) and a mobile humanoid platform (T.B.D). Whole-body manipulation is one of the most rapidly emerging research areas globally. To lead advancements in this domain, the RIRO Lab focuses on both novel policy learning methods based on imitation & reinforcement learning, as well as high-level planning strategies leveraging large language models (LLMs) and other foundation models. Our goal is to develop hierarchical foundation models for complex task executions. We will evaluate the proposed methods on our in-house humanoid platforms and through collaborations with external partners, including Hyundai Motor Company and Samsung Electronics.

Keywords: Imitation learning, State-space models (SSM), Diffusion policy, Constraint learning, humanoid navigation
Selected paper: [under review]

In-hand manipulation of deformable objects offers unprecedented opportunities to resolve various real-world problems, such as binding and taping. This project aims to develop a visuotactile in-hand manipulation that repositions/reorientations deformable objects in hand as we want. Toward this line of research, we propose three research thrusts: 1) a physics-informed reinforcement learning (RL) framework, 2) an interactive RL framework, and 3) Sim2Real transfer learning method.

Keywords: (Inverse) Reinforcement learning, Deformable obejct manipulation, Sim2Real transfer learning,
Selected paper: [CoRL19], [RA-L23]

We aim to introduce task-and-motion planning (TAMP) framework that is to solve complex and longer-time horizon of human tasks. To resolve completeness, optimality, and robustness issues, we are working on various task planning and motion planning approaches. We will show a generalizable TAMP framework under human operator’s cooperative or adversarial interventions.

Keywords: Large language models, Large multimodal models, Semantic perception, Behavior tree
Selected paper: [ICRA21], [RA-L22], [ICRA24]

Natural language is a convenient means to deliver a user’s high-level instruction. We introduce a language-guided manipulation framework that learns common-sense knowledge from natural language instructions and corresponding motion demonstrations. We apply the technologies on various quadrupedal robots like Boston Dynamics Spot!

Keywords: Quadruped robot, Semantic SLAM, Natural language grounding, Space grounding
Selected paper: [IJRR20], [FR22], [AAAI24]