ManifoldRG / NEKO_Archive

The NEKO Project is an open source effort to build a model of equivalent scale and capability as that reported in DeepMind’s 2022 Paper, A Generalist Agent .
GNU General Public License v3.0
10 stars 1 forks source link

Sourcing a Control Dataset #13

Closed daniellawson9999 closed 1 year ago

daniellawson9999 commented 1 year ago

Purpose

This issue is created to describe the process for suggesting and completing the sourcing of a control dataset. Inspiration for picking datasets to source can be derived from https://github.com/orgs/ManifoldRG/projects/12?pane=issue&itemId=28151504, or other places.

Background

The first discussion for control datasets can be found here: https://github.com/orgs/ManifoldRG/projects/12?pane=issue&itemId=28151504

The result of this discussion is that we will use the Minari format for storing datasets, as well as Gymnasium, which uses the new v26 environment format. I would recommend reading through the Minari documentation.

We recommended that a good pair of initial environments are the MuJoCo Locomotion and Atari tasks. As of the writing of this comment, the porting for MuJoCo is complete, and the progress for Atari can be found here: https://github.com/orgs/ManifoldRG/projects/12?pane=issue&itemId=28733001

Sourcing a new Dataset

To source a new dataset, create an issue regarding the dataset. In the issue, describe the paper(s), repo(s), and other relevant information that has information regarding the dataset. There may be other relevant information to include, such as the RL algorithm that was used to collect the data or the human, number of trajectories/episodes.

There are two important pieces to solve regarding adding a new dataset: 1) The main important piece is that the environment either already supports the new Gymnasium API, already has a conversion implemented, such as in Shimmy, or we can port to the new API ourselves. 2) Data associated with the environment must be converted to the Gymnasium format. This is mostly accomplished using the https://minari.farama.org/api/minari_dataset/minari_dataset/#minari.MinariDataset.update_dataset_from_buffer function, which is demonstrated in https://github.com/daniellawson9999/data-tests/blob/main/d4rl_mujoco_minari/mujoco_pkl_to_minari.py along with other steps.

The issue should track the progress of both tasks 1), 2), and potentially can branch to other issues, tracking individual items, but the result of all work should be described in a concluding comment on this main issue. Once an environment is converted to gymnasium, and its data is imported to Minari, we can hopefully seamlessly plug it in, with minimal code changes.

BobakBagheri commented 1 year ago

Issue is closed, Leaving here as reference on how to make new issues, expectations on how to write issues, etc.