search

tensorflow / tensor2tensor

Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.
Apache License 2.0
15.4k stars 3.48k forks source link

*question* would a real robotics dataset be interesting for the T2T community? #839

Open mariabauza opened 6 years ago

mariabauza commented 6 years ago

Description

Hi,

I am a PhD student at MIT and I wanted to know if the T2T community would be interested in a robotic dataset that contains data from real experiments rather than simulation.

Overview of the dataset: We provide a comprehensive and high-fidelity dataset of planar pushing experiments. The dataset contains timestamped poses of a pusher and a pushed object, as well as forces at the interaction. The push interaction varies in 6 dimensions: surface material, shape of the pushed object, contact position, pushing direction, pushing speed, and pushing acceleration.

We could also provide a dataset that contains both the object an pusher positions together with videos of the interactions. This could be easily used in combination with deep learning techniques for computer vision.

If you believe there is value, we would love to make the effort to submit this datasets to T2T. Please send us feedback. #

rsepassi commented 6 years ago

Yes, that'd be great!

Are there torques or actions of some kind in the dataset too? Would be useful for RL applications.

Code pointers that may be helpful:

mariabauza commented 6 years ago

Hi rsepassi,

We use an ABB IRB 120 industrial robotic arm with 6 DOF that can control precisely the position, velocity, and acceleration of its tool center point (TCP). Thus the actions that we consider in the dataset are position, velocity, and acceleration of the TCP.

The dataset would include the following signals:

  1. Position, velocity, and acceleration of the TCP
  2. Force-torque recordings between the TCP and the pushed object
  3. Object position and orientation
  4. (For some cases) RGBD video of the interaction.

To have a better flavor, how large do datasets have to be in order to be useful for the community? Thanks!