search

leolyliu / TACO-Instructions

Official repository of "TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understanding".
48 stars 1 forks source link

dataset format #1

Open zhoubohan0 opened 5 months ago

zhoubohan0 commented 5 months ago

I'd like to run visualization.py but surprisingly found that the pre-released dataset doesn't match the codes.

  1. tool_model should be in format join(object_model_root, target_name + "_cm.obj") and hand_pose directory should be in format join(dataset_root, "Hand_Poses", triplet, sequence_name). Could you please make some updates or give any friendly explanations?
  2. current pickle files are not readable. Is there any solutions?
  3. currently the whole dataset only include videos. Will other pose information be added in the future?
zhoubohan0 commented 5 months ago

BTW, an email was delivered. Look for your reply!

leolyliu commented 5 months ago

Hi, zhoubohan0:

For hand-object motion visualization, please use the command in "README.md": python visualization.py --dataset_root <dataset root directory> --object_model_root <object model root directory> --triplet <triplet name> --sequence_name <sequence name> --save_path <path to save the visualization result> --device <device for the rendering process>

The <dataset root directory> is the root directory, which includes subfolders named "Hand_Poses", "Object_Poses", "Object_Models", "Allocentric_RGB_Videos", "Egocentric_RGB_Videos", "Egocentric_Depth_Videos". The <object model root> is the path of the "Object_Models" folder, including 206 OBJ files.

Thank you for your feedback, we will add the data organization information in "README.md".

leolyliu commented 5 months ago

The pickle files can be loaded using the following command:

import pickle
data = pickle.load(open("left_hand.pkl", "rb"))
leolyliu commented 5 months ago

The allocentric videos and hand-object poses of the whole dataset are post-processing. We will release them soon.