The NEKO Project is an open source effort to build a model of equivalent scale and capability as that reported in DeepMind’s 2022 Paper, A Generalist Agent .
The result of this discussion is that we will use the Minari format for storing datasets, as well as Gymnasium, which uses the new v26 environment format. I would recommend reading through the Minari documentation.
To source a new dataset, create an issue regarding the dataset. In the issue, describe the paper(s), repo(s), and other relevant information that has information regarding the dataset. There may be other relevant information to include, such as the RL algorithm that was used to collect the data or the human, number of trajectories/episodes.
The issue should track the progress of both tasks 1), 2), and potentially can branch to other issues, tracking individual items, but the result of all work should be described in a concluding comment on this main issue.
Once an environment is converted to gymnasium, and its data is imported to Minari, we can hopefully seamlessly plug it in, with minimal code changes.
Purpose
This issue is created to describe the process for suggesting and completing the sourcing of a control dataset. Inspiration for picking datasets to source can be derived from https://github.com/orgs/ManifoldRG/projects/12?pane=issue&itemId=28151504, or other places.
Background
The first discussion for control datasets can be found here: https://github.com/orgs/ManifoldRG/projects/12?pane=issue&itemId=28151504
The result of this discussion is that we will use the Minari format for storing datasets, as well as Gymnasium, which uses the new v26 environment format. I would recommend reading through the Minari documentation.
We recommended that a good pair of initial environments are the MuJoCo Locomotion and Atari tasks. As of the writing of this comment, the porting for MuJoCo is complete, and the progress for Atari can be found here: https://github.com/orgs/ManifoldRG/projects/12?pane=issue&itemId=28733001
Sourcing a new Dataset
To source a new dataset, create an issue regarding the dataset. In the issue, describe the paper(s), repo(s), and other relevant information that has information regarding the dataset. There may be other relevant information to include, such as the RL algorithm that was used to collect the data or the human, number of trajectories/episodes.
There are two important pieces to solve regarding adding a new dataset: 1) The main important piece is that the environment either already supports the new Gymnasium API, already has a conversion implemented, such as in Shimmy, or we can port to the new API ourselves. 2) Data associated with the environment must be converted to the Gymnasium format. This is mostly accomplished using the https://minari.farama.org/api/minari_dataset/minari_dataset/#minari.MinariDataset.update_dataset_from_buffer function, which is demonstrated in https://github.com/daniellawson9999/data-tests/blob/main/d4rl_mujoco_minari/mujoco_pkl_to_minari.py along with other steps.
The issue should track the progress of both tasks 1), 2), and potentially can branch to other issues, tracking individual items, but the result of all work should be described in a concluding comment on this main issue. Once an environment is converted to gymnasium, and its data is imported to Minari, we can hopefully seamlessly plug it in, with minimal code changes.