search

vikashplus / robohive

A unified framework for robot learning
https://sites.google.com/view/robohive
Apache License 2.0
489 stars 82 forks source link

Dataset Description for FK1-v4(human). #143

Closed bgstn closed 4 months ago

bgstn commented 4 months ago

Hi,

Thank you for this really great repos. This is the most easily and stable of all the environments I have used so far. Really appreciate what you have been doing here!!

I just want to ask if there is any description for observations of the dataset linked here: https://drive.google.com/drive/folders/1a-q6TpskJD3J7G2FcJBzYRb7UxG1N_K4.

I am trying to use the FK1-v4(human) dataset for my model. I downloaded the dataset in the Google Drive in the wiki linked above, I found there are some mismatches between the dataset in the original repo. https://github.com/google-research/relay-policy-learning and the one in the Google Drive.

The observation is 60 dimensions in the original dataset while the one in Google Drive is 75 dimensions. I found the description for 59 of the dimensions in the gymnasium(https://robotics.farama.org/envs/franka_kitchen/franka_kitchen/) and the 60th is the time step, which I assume to be the first 60 dimensions out of 75 in the FK1-v4(human). Then what are the descriptions for the rest 15 dimensions?

I have browsed a while and couldn't find any useful information.

Any help would be greatly appreciated!

Best Regards Juyan

vikashplus commented 4 months ago

I'm glad you are enjoying working with RoboHive. I understand the confusion here. The documentation is sparse.

The goal of RoboHive is to bring together several disjoint pieces of work and make them compatible. The newly released dataset (e.g FK1-v4(human)) is compatible with the updated Robohive envs. You can read here details on the evolution of the envs and bug fixes.

Now, to help with your question. I have updated the readMe with details. Let me know if it helps. Please reopen the issue if you have more questions.

bgstn commented 4 months ago

@vikashplus Thank you for the help. I appreciate it.

Based on the details and readMe. I found the description for the 75 dimensions. It is nothing like I described earlier. I attached it below for future reference.

They are obot_jnt: (9,) objs_jnt: (15,) obj_goal:(15,) ee_pose:(6,) knob1_site_err:(3,) knob2_site_err:(3,) knob3_site_err:(3,) knob4_site_err:(3,) light_site_err:(3,) slide_site_err:(3,) leftdoor_site_err:(3,) rightdoor_site_err:(3,) microhandle_site_err:(3,) kettle_site0_err:(3,)

However, I found that there seems to be no task information in the data. But for the original dataset, the data directory name has the sequence of the goals the end effector tries to reach. I want to build some task representation with task information.

So is there any task information in the data? Best Regards Juyan

vikashplus commented 4 months ago

I'd encourage you to read the env change log, especially between v2->v3 to understand the motivation behind the new observation space and how it is made task agnostic to better help with generalization studies without leaking information (via the difference in observation space). With this change, there is no difference between observation and tasks. Its a true multi-task goal-conditioned play data.

Yet for any reason, the old information is still relevant, here are few things to try