IRIS: An Immersive Robot Interaction System

Xinkai Jiang1,*, Qihao Yuan2, Enes Ulas Dincer1, Hongyi Zhou1, Ge Li1, Xueyin Li1, Julius Haag1, Nicolas Schreiber1, Kailai Li2, Gerhard Neumann1, Rudolf Lioutikov1,
1 Karlsruhe Institute of Technology
2 University of Groningen
*Corresponding Author
Code (Coming Soon) arXiv

Abstract

This paper introduces IRIS, an Immersive Robot Interaction System leveraging Extended Reality (XR). IRIS was designed for immersive robot data collection and interaction across multiple simulators, benchmarks, and real-world scenarios. While existing XR-based data collection systems provide efficient and intuitive solutions for large-scale data collection, they are often challenging to reproduce and reuse. This limitation arises because current systems are highly tailored to simulatorspecific use cases and environments. IRIS is a novel, easily extensible framework that already supports multiple simulators, benchmarks, and even headsets. Furthermore, IRIS is able to include additional information from real-world sensors, such as point clouds captured through depth cameras. An unified scene specification is generated directly from simulators or realworld sensors and transmitted to XR headsets, creating identical scenes in XR. This specification allows IRIS to support any of the objects, assets, and robots provided by the simulators. In addition, IRIS introduces shared spatial anchors and a robust communication protocol that links simulations between multiple XR headsets. This feature enables these XR headsets to share a synchronized scene, facilitating collaborative and multiuser data collection. IRIS can be deployed on any device that supports the Unity framework, encompassing the vast majority of commercially available headsets. In this work, IRIS was deployed and tested on the Meta Quest 3 and the HoloLens 2. IRIS showcased its versatility across a wide range of real-world and simulated scenarios, using current popular robot simulators such as MuJoCo, IsaacSim, CoppeliaSim, and Genesis. In addition, a user study evaluates IRIS on a data collection task for the LIBERO benchmark. The study shows that IRIS significantly outperforms the baseline in both objective and subjective metrics. This paper demonstrates the huge potential that IRIS provides for the future of data collection and interaction in the robotics and robot learning community.

IRIS supports different simulators and real robots.

Comparison of IRIS and other data collection systems.

BibTeX

@misc{jiang2025irisimmersiverobotinteraction,
        title={IRIS: An Immersive Robot Interaction System}, 
        author={Xinkai Jiang and Qihao Yuan and Enes Ulas Dincer and Hongyi Zhou and Ge Li and Xueyin Li and Julius Haag and Nicolas Schreiber and Kailai Li and Gerhard Neumann and Rudolf Lioutikov},
        year={2025},
        eprint={2502.03297},
        archivePrefix={arXiv},
        primaryClass={cs.RO},
        url={https://arxiv.org/abs/2502.03297}, 
  }