PRISM Precision and contact-rich Real-world Industrial Skill Dataset with Multimodal Sensing

A large-scale multimodal dataset for contact-rich real-world
industrial manipulation.

Tengbo Yu1,2, Jiahao Wu2, Hanning Wang1, Rui Chen3, Chuanhou Liu4, Chuang Sun5, Hangxin Liu1†

1State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University
2Delta Intelligence
3PKU-Wuhan Institute for Artificial Intelligence

4Hubei Humanoid Robot Innovation Center Co., Ltd
5China Academy of Information and Communications Technology


01

Abstract

Recent progress in robotic learning has been fueled by large-scale datasets collected in everyday environments. However, most existing datasets emphasize short-horizon, low-contact tasks such as pick-and-place, and therefore do not capture the precision control, force/torque or tactile regulation, and multimodal feedback required for industrial assembly. To address this gap, we introduce PRISM, a large-scale multimodal dataset for contact-rich industrial operations.

The dataset spans more than 25 manipulation tasks, including electronic component plug/unplug and conveyor-based sorting, and covers diverse mechanical constraints. PRISM includes more than 5,000 robot trajectories with paired human demonstrations, totaling over 45 hours of data, recorded using synchronized multi-view RGB-D, force/torque, tactile, and robot-state measurements. PRISM provides a realistic benchmark for multimodal perception and control under high-precision industrial constraints, and serves as a foundation for contact-rich, generalizable manipulation in real-world manufacturing environments.

02

Data Volume

25+ Industrial Tasks
5,000+ Robot Trajectories
5,000+ Human Demonstrations
45+ Hours
~27M Images
RGB-D Multi-view Vision
6DoF F/T Contact Sensing
Tactile + Proprioception
03

Overview

Overview of the PRISM multimodal industrial manipulation dataset
Fig. 1. Overview of PRISM, a multimodal dataset for contact-rich real-world industrial manipulation.

PRISM is designed for high-precision industrial manipulation where contact, force regulation, and multimodal feedback are essential. The dataset covers multi-robot embodiments, multi-view observations, multimodal sensing streams, and diverse industrial skills. Each episode is recorded with synchronized visual, tactile, force/torque, and proprioceptive signals, enabling the study of contact-rich manipulation beyond short-horizon, low-contact tasks.

04

Data Collection Platform

Three teleoperation platforms used to collect the PRISM dataset
Fig. 2. Illustration of the PRISM data collection platforms with multi-view RGB-D cameras.

PRISM collects demonstrations using three complementary teleoperation interfaces: exoskeleton-based control, tracker-based control, and VR-based control. These platforms introduce different human control styles and reduce the bias of any single collection setup. Across the platforms, PRISM records robot states, multi-view RGB-D images, force/torque measurements, tactile observations, and gripper states with timestamps for multimodal alignment.

05

Video

06

BibTeX

coming soon