search

RLE-Foundation / RLeXplore

RLeXplore provides stable baselines of exploration methods in reinforcement learning, such as intrinsic curiosity module (ICM), random network distillation (RND) and rewarding impact-driven exploration (RIDE).
https://docs.rllte.dev/
MIT License
367 stars 15 forks source link

atari enviroment error #20

Closed haneenhassen closed 3 months ago

haneenhassen commented 3 months ago

When executing the following code: python

from stable_baselines3.common.env_util import make_vec_env, make_atari_env from stable_baselines3.common.vec_env import VecFrameStack

envs = make_atari_env("MontezumaRevenge-v4") envs = VecFrameStack(envs, n_stack=4)

print(envs.observation_space, envs.action_space)

the output is:

Box(0, 255, (84, 84, 4), uint8) Discrete(18)

This indicates that the observation space is a Box with shape (84, 84, 4).

In contrast, when executing this code: python

from rllte.env import make_atari_env

envs = make_atari_env("MontezumaRevenge-v5") print(envs.observation_space, envs.action_space)

the output is:

Box(0, 255, (4, 84, 84), uint8) Discrete(18)

Here, the observation space has been transformed to a Box with shape (4, 84, 84).

yuanmingqi commented 3 months ago

This is because an ImageTranspose wrapper has been used by default in rllte.

You can refer to https://github.com/vwxyzjn/cleanrl/blob/master/cleanrl/ppo_atari.py to use a proper Atari wrapper.

And you need to use (4, 84, 84), i.e., channel first, for rllte's intrinsic reward modules.

haneenhassen commented 3 months ago

thanks