takuseno / d4rl-atari

Datasets for data-driven deep reinforcement learning with Atari (wrapper for datasets released by Google)
MIT License
101 stars 14 forks source link

How are actions in the dataset defined? #16

Open hutchinsonian opened 3 months ago

hutchinsonian commented 3 months ago

To be more specific, in the ms-pacman environment, I use env = gym.make('ms-pacman-medium-v0') to create the env, and I run

observation = env.reset() # observation.shape == (84, 84)
observation, reward, terminal, info, _ = env.step(2)

I found that the action definition here is the same as https://gymnasium.farama.org/environments/atari/ms_pacman/ , action 2 controls the character to move to the right. But in data = env.get_dataset(), I found that there seems to be some difference.

data['actions'][:10]
array([2, 2, 2, 2, 6, 2, 2, 7, 7, 7], dtype=int32)

I saved the first few frames of observation: image image image image

Why is the character moving upwards? Am I missing something? @takuseno @takuyamagata