[Proposal] Humanoid Offline RL data

[prajjwal1]

Proposal I was wondering if there's interest in making humanoid part of D4RL. The dataset was introduced in the following paper

Paper: https://arxiv.org/abs/2305.14550

The link for the data is here: https://dl.fbaipublicfiles.com/prajj/rl_paradigm/humanoid_offline_rl_data.tar.gz Repo : https://github.com/prajjwal1/rl_paradigm

Motivation Making humanoid available would be helpful for offline RL community for the same reason existing datasets are. Humanoid is more challenging in some ways than existing D4RL datasets such as state space dimension.

We provide medium, medium-expert and expert data for humanoid all in the same format as D4RL. It contains a lot of timestep data than what existing D4RL datasets provide.

[prajjwal1]

https://github.com/Farama-Foundation/D4RL/issues/221

[rodrigodelazcano]

Hey @prajjwal1 thank you for sharing this. We would definitely like to incorporate these new datasets to Minari. Although, I don't think we would be able to add them to D4RL since we are trying not to make any updates to the source code for legacy purposes and continue development/standardization in Minari.

We are also working on migrating the raw D4RL datasets with their original Gym environment. However, our work at the moment has been focused on recreating these datasets and standardizing them with the updated environments that follow the Gymnasium API. (We are keeping the dataset generation scripts in this repo https://github.com/rodrigodelazcano/d4rl-minari-dataset-generation, which will be transferred to Farama).

Would you be interested and do you have the time to recreate these datasets for Minari? (The environment should be the one in Gymnasium, we haven't yet added functionality to create datasets from Gym environments in Minari)

[prajjwal1]

Thanks for replying @rodrigodelazcano. Yes. I can add humanoid inside minari. If there's a example code that you can point me to, I can add humanoid. It should be identical as importing other d4rl tasks, since the dict files i created follow the D4RL convention.

[rodrigodelazcano]

Thank you! Sure, for doing a direct migration from a d4rl dataset to minari the adroit datasets are the best examples (https://github.com/rodrigodelazcano/d4rl-minari-dataset-generation/blob/main/scripts/adroit/recreate_adroit_door.py). However, in my opinion it would be best to generate the datasets from scratch using the data collection method you used such as in the pointmaze datasets (https://minari.farama.org/tutorials/dataset_creation/point_maze_dataset/).

Once you have a script that generates the minari datasets for humanoid you can share it with us an we will generate the datasets on our end and upload them to the server. We are adding the generation scripts in this repo: https://github.com/rodrigodelazcano/d4rl-minari-dataset-generation. You can make a PR to this repo with the generation code or add the code to another repo of your own and we will make a reference to it inside the datasets.