search

google-deepmind / open_x_embodiment

Apache License 2.0
623 stars 41 forks source link

Depth images #53

Closed RussRobin closed 3 months ago

RussRobin commented 3 months ago

Hi, thank you for your great work.

For datasets with depth cams (e.g.), Berkeley Bridge and Berkeley Autolab UR5, depth images are included in their original datasets.

In RT-X dataset, I can find Berkeley Autolab UR5's depth images in ['observation']['image_with_depth'], but I cannot find depth image in Berkeley Bridge. I wonder how do RT-X process depth images for each dataset?

Thanks in advance!

kpertsch commented 3 months ago

For RT-X, we don't train on depth images. Many datasets still include the depth images though if you want to train on them. For bridge specifically, I believe only a small part of the original data actually has depth images, so we did not include them in our RLDS version of the dataset. If you want to train on them, please refer to the original dataset or you can build off of this conversion repo to re-process the RLDS data and include the depth images!

RussRobin commented 3 months ago

Thank you for your timely reply! I will close the issue.