Port Atari to Minari

[daniellawson9999]

Context

In #4 , we proposed starting with Atari and D4RL MuJoCo datasets. I have experimented with porting the D4RL MuJoCo dataset to Minari at : https://github.com/daniellawson9999/data-tests

Goal

Similarly, port https://github.com/takuseno/d4rl-atari / https://research.google/resources/datasets/dqn-replay/ , paper to Minari . Also need to track and match configuring the new gymnasium environments (v5) to match the settings in the paper.

[daniellawson9999]

progress ongoing here: https://github.com/daniellawson9999/data-tests

[daniellawson9999]

Porting has been tested using the expert subset on a few datasets, and documentation can be found in the https://github.com/daniellawson9999/data-tests Port Atari to Minari section.

A next step is running this over all environments, and storing datasets in the cloud. Could copy Minari upload/download logic, but using our own bucket/other service: https://github.com/Farama-Foundation/Minari/blob/main/minari/storage/hosting.py

[harshsikka]

@daniellawson9999 any other updates on this? Should this be closed?

[AmrMKayid]

@harshsikka @daniellawson9999 Have you explored this repo instead of Minari? https://takuseno.github.io/d3rlpy/

I think it might be better since it contains Atari, MuJoCo, and PyBullet

[daniellawson9999]

Yeah, I have seen d3rlpy before but I believe it has several significant disadvantages over Minari:

Library / dataset design: When I reviewed their implementation, we would handle problems related to memory as I discussed in issue https://github.com/ManifoldRG/NEKO/issues/4. I believe d3rl directly loads episodes into memory. Minari allows us to load many datasets, which we will require (say 100 control environments), but only pulls episodes into memory after a sample_episodes call. I would also recommend reading through both documentation, as Minari's is much cleaner and d3rl is lacking a lot of info.

Development: Minari has a large amount of active development, and is up to date with recent gymnasium standards.d3rl is using Gym==0.17. It would be much easier to integrate newer environments into Minari as it is up to date.

It is also easy to port datasets to Minari, as demonstrated I have already adapted both Atari and MuJoCo to Minari so these datasets are not a problem. In addition to these datasets, Minari offers several d4rl robotics tasks which are pretty good: https://minari.farama.org/datasets/.

In general, Minari does not just provide a source of datsets, although I believe it also is better in this regard, but also as a system for us to manage our datasets. There is a large community effort to add more datasets to Minari, so will get a lot of datasets for free, as well. This blog has some good info: https://farama.org/Announcing-Minari)

[BobakBagheri]

Finished and closing.