Farama-Foundation / D4RL

A collection of reference environments for offline reinforcement learning
Apache License 2.0
1.29k stars 279 forks source link

"timeouts" is missing in maze2d environments #176

Open takuma-yoneda opened 1 year ago

takuma-yoneda commented 1 year ago

When I download a maze2d dataset with env.get_dataset(), the downloaded hdf5 file contains "timeouts" key:

>>> import d4rl, gym
>>> env = gym.make('maze2d-large-v1')
>>> dataset = env.get_dataset()
Downloading dataset: http://rail.eecs.berkeley.edu/datasets/offline_rl/maze2d/maze2d-large-sparse-v1.hdf5 to /root/.d4rl/datasets/maze2d-large-sparse-v1.hdf5
load datafile: 100% 8/8 [00:01<00:00,  6.41it/s]
>>> dataset.keys()
dict_keys(['actions', 'infos/goal', 'infos/qpos', 'infos/qvel', 'observations', 'rewards', 'terminals', 'timeouts'])

However, when I attempt to generate the same dataset locally, it doesn't contain "timeouts" key:

$ python scripts/generation/generate_maze2d_datasets.py --env_name maze2d-large-v1
$ python scripts/generation/relabel_maze2d_rewards.py --relabel_type sparse --maze large --filename /data/maze2d/maze2d-large-v1.hdf5
# ^ This generates maze2d-large-v1-sparse.hdf5
$ python
>>> import d4rl, gym
>>> env = gym.make('maze2d-large-v1')
>>> dataset = env.get_dataset('/data/maze2d/maze2d-large-sparse-v1.hdf5')  # Load from a local file
load datafile: 100% 7/7 [00:00<00:00, 21.77it/s]
>>> dataset.keys()
dict_keys(['actions', 'infos/goal', 'infos/qpos', 'infos/qvel', 'observations', 'rewards', 'terminals'])  # <-- This is missing "timeouts"!

I believe I followed the wiki page to generate the dataset. Am I missing any steps?

Liang-ZX commented 1 year ago

When I download a maze2d dataset with env.get_dataset(), the downloaded hdf5 file contains "timeouts" key:

>>> import d4rl, gym
>>> env = gym.make('maze2d-large-v1')
>>> dataset = env.get_dataset()
Downloading dataset: http://rail.eecs.berkeley.edu/datasets/offline_rl/maze2d/maze2d-large-sparse-v1.hdf5 to /root/.d4rl/datasets/maze2d-large-sparse-v1.hdf5
load datafile: 100% 8/8 [00:01<00:00,  6.41it/s]
>>> dataset.keys()
dict_keys(['actions', 'infos/goal', 'infos/qpos', 'infos/qvel', 'observations', 'rewards', 'terminals', 'timeouts'])

However, when I attempt to generate the same dataset locally, it doesn't contain "timeouts" key:

$ python scripts/generation/generate_maze2d_datasets.py --env_name maze2d-large-v1
$ python scripts/generation/relabel_maze2d_rewards.py --relabel_type sparse --maze large --filename /data/maze2d/maze2d-large-v1.hdf5
# ^ This generates maze2d-large-v1-sparse.hdf5
$ python
>>> import d4rl, gym
>>> env = gym.make('maze2d-large-v1')
>>> dataset = env.get_dataset('/data/maze2d/maze2d-large-sparse-v1.hdf5')  # Load from a local file
load datafile: 100% 7/7 [00:00<00:00, 21.77it/s]
>>> dataset.keys()
dict_keys(['actions', 'infos/goal', 'infos/qpos', 'infos/qvel', 'observations', 'rewards', 'terminals'])  # <-- This is missing "timeouts"!

I believe I followed the wiki page to generate the dataset. Am I missing any steps?

Excuse me. I meet the same question. Have you solved this problem? Thank you.