Initialize with motor babbling
The robot explores joint motions while wearing the exoskeleton. Encoder readings, short motion histories, and robot joint positions train the initial encoder-to-command mapping.
Conference on Robot Learning CoRL 2026
SEED-UMISharing the Exoskeleton between human and robot for onE-to-one Dexterous demonstration
1 State Key Laboratory of General Artificial Intelligence,
School of Intelligence Science and Technology, Peking University
2 Delta Intelligence
01 / Overview
One exoskeleton.
A shared physical interface.
What if the human demonstrator and the robot could share the same measurement interface?
SEED-UMI makes this possible by letting both wear the same exoskeleton. Joint encoders record corresponding movements, while cameras fixed to the exoskeleton observe the same outer mechanism during human data collection and robot execution.
This turns motion retargeting into paired cross-embodiment supervision. Contact-rich human demonstrations become training data for dexterous robot policies, directly from raw wrist images.
The project film
SEED-UMI in action.
From natural human demonstrations
to autonomous dexterous manipulation.
02 / Shared hardware
The same structure.
On both sides.
The human and robot share the outer exoskeleton, joint axes, fingertip pads, and sensor placement. Cuffs and mounts adapt the interface to each embodiment.
20 DoF
One-to-one joint correspondence
Twenty independently measured joints across five fingers, matched to the fully actuated target hand.
6 DoF
Wrist motion tracking
A dorsal Intel RealSense T265 records the wrist trajectory in the exoskeleton frame.
150°
Exoskeleton-centric vision
A ventral fisheye camera observes the workspace from a matched viewpoint during demonstration and execution.
Designed for contact
Keep the contact surfaces clear.
A parallel four-bar linkage routes joint rotation to dorsally mounted magnetic encoders. This preserves angular correspondence while keeping the lateral and palmar surfaces clear for manipulation.
03 / Method
From shared measurements
to executable actions.
Two-stage motion mapping.
Direct learning from human demonstrations.
Refine with paired replay
Human contact-rich motions are replayed on the robot. Differences between the two encoder trajectories provide supervision for correcting the mapping under contact.
Learn and deploy the policy
ACT, Diffusion Policy, and π₀.₅ learn wrist–hand action sequences from raw wrist images and encoder states. The refined mapping converts predicted encoder targets into robot commands.
Policy learning uses human-collected real data, without simulation, reinforcement learning, hand segmentation, or generative inpainting.
04 / Experiments
Five tasks. Rich contact.
Real-world execution.
Evaluated on a RealMan RX75 arm and a 20-DoF Wuji hand, with 100 demonstrations per task and 20 autonomous rollouts for each task, policy, and condition.
Screw Driving
Align a screwdriver and maintain a stable grasp and axial pressure.
AirPods Case Insertion
Coordinate the index and middle fingers for small-object placement.
Ball Basket Throwing
Time the transition from grasping to a dynamic release.
Table Cleaning
Extract a tissue, wipe under contact, and discard it.
Air Freshener Spray
Stabilize the can and initiate a downward actuator press.
Results at a glance
Paired supervision closes much of the transfer gap.
70.0%
Mean rollout success
Across five tasks and three policy backbones.
+12.7pp
Gain from paired fine-tuning
Compared with the babbling-only mapping.
~3×
Collection throughput
52 vs. 18 successful AirPods demonstrations in 30 minutes.
Mean success is 71.7% for teleoperation, 57.3% for SEED-UMI with the babbling-only mapping, and 70.0% after paired fine-tuning. SEED-UMI trains its policies on human-side demonstrations; the teleoperation baseline uses robot-side demonstrations.
| Policy | Condition | Screw Driving | AirPods Case | Ball Throwing | Table Cleaning | Air Freshener | Mean |
|---|---|---|---|---|---|---|---|
| ACT | Teleoperation | 80.0 | 65.0 | 60.0 | 75.0 | 90.0 | 74.0 |
| ACT | SEED-UMI · babbling only | 55.0 | 65.0 | 60.0 | 50.0 | 70.0 | 60.0 |
| ACT | SEED-UMI · paired fine-tuning | 70.0 | 75.0 | 70.0 | 65.0 | 85.0 | 73.0 |
| Diffusion Policy | Teleoperation | 65.0 | 55.0 | 50.0 | 60.0 | 70.0 | 60.0 |
| Diffusion Policy | SEED-UMI · babbling only | 40.0 | 55.0 | 50.0 | 40.0 | 45.0 | 46.0 |
| Diffusion Policy | SEED-UMI · paired fine-tuning | 55.0 | 60.0 | 55.0 | 55.0 | 60.0 | 57.0 |
| π₀.₅ | Teleoperation | 90.0 | 75.0 | 70.0 | 85.0 | 85.0 | 81.0 |
| π₀.₅ | SEED-UMI · babbling only | 60.0 | 75.0 | 70.0 | 60.0 | 65.0 | 66.0 |
| π₀.₅ | SEED-UMI · paired fine-tuning | 80.0 | 85.0 | 80.0 | 75.0 | 80.0 | 80.0 |
Evaluation details. Initial object positions are randomized. Paired mapping calibration uses 250 episodes across five objects. Air Freshener Spray success requires a secure grasp and an initiated downward press; full actuator depression and spray emission are not required. The throughput comparison uses the same operator in separate 30-minute windows and excludes one-time paired-replay and mapping-optimization overhead.
05 / Closer look
Why the collection
interface matters.
Direct feedback for delicate contact.
During AirPods insertion, the operator interacts directly with the object and can regulate contact through immediate visual and physical feedback. The qualitative comparison below illustrates human and robot encoder traces, alongside a teleoperation replay that over-drives the contact.
Collection efficiency
More demonstrations.
Comparable policy performance.
The human can collect demonstrations independently of real-time robot execution. In the paper’s AirPods study, SEED-UMI records 52 successful demonstrations in 30 minutes, compared with 18 using teleoperation.
After accounting for the reported overall rollout success, the paper estimates approximately 2.9× as much useful training data per minute, excluding one-time calibration overhead.
Looking ahead
Toward more scalable
dexterous demonstration.
The current exoskeleton is co-designed for one target hand. Adapting to other hands still requires changes to geometry, calibration motions, and mapping. Future directions include automating that adaptation, improving wearability, and adding tactile sensing for more precise contact estimation.
Citation
@misc{yu2026seedumisharingexoskeletonhuman,
title={SEED-UMI: Sharing the Exoskeleton between human and robot for onE-to-one Dexterous demonstration},
author={Tengbo Yu and Jiahao Wu and Daohan Li and Bingxu Chen and Hao Liu and Xiaojian Ma and Hangxin Liu},
year={2026},
eprint={2609.11753},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2609.11753},
}