FrameStack Bug.

[Liuzy0908]

Bug description

I want to use the frame stacking technique (4 consecutive frames of images as model input), which works well in PPO-only in SB3.

But after running the above program (about GAIL), the shape of obs collected by rollouts is: (1024,59,256,1). It seems to be collected are single frame images instead of 4 consecutive frames. I think the correct shape of obs should be (1024,59,256,4).

This causes the policy network to fail later when GAIL.train(): The obs input of PPO is (batch, 4, 59, 256), while the obs provided by rollouts is (batch, 1, 59, 526).

How should I solve this problem? Looking forward to your reply.

Steps to reproduce

from stable_baselines3.common.vec_env import VecFrameStack, DummyVecEnv

print(MyEnv().observation_space.shape)              # out: (59, 256, 1). 

venv = DummyVecEnv([lambda: RolloutInfoWrapper(MyEnv())])
env = VecFrameStack(venv, n_stack=4)

print(env.observation_space.shape)                  # out: (59, 256, 4). Frame Stack.

expert = PPO.load(model_dir, env=env)

print(expert.get_env().observation_space.shape)     # out: (4, 59, 256)

rng = np.random.default_rng()
rollouts = rollout.rollout(
    expert,
    expert.get_env(),
    rollout.make_sample_until(min_timesteps=None, min_episodes=1),
    rng=rng,
)

Environment

• Operating system and version: Windows 10
• Python version: 3.8
• Output of pip freeze --all:absl-py==1.4.0 airsim==1.8.1 cachetools==5.3.0 certifi @ file:///C:/b/abs_85o_6fm0se/croot/certifi_1671487778835/work/certifi chai-sacred==0.8.3 charset-normalizer==3.1.0 cloudpickle==2.2.1 colorama==0.4.6 contourpy==1.0.7 cycler==0.11.0 docopt==0.6.2 filelock==3.9.1 fonttools==4.39.0 gitdb==4.0.10 GitPython==3.1.31 google-auth==2.16.2 google-auth-oauthlib==0.4.6 grad-cam==1.4.6 grpcio==1.51.3 gym==0.21.0 huggingface-hub==0.13.2 huggingface-sb3==2.2.4 idna==3.4 imitation==0.3.2 importlib-metadata==4.13.0 importlib-resources==5.12.0 joblib==1.2.0 jsonpickle==3.0.1 kiwisolver==1.4.4 Markdown==3.4.1 markdown-it-py==2.2.0 MarkupSafe==2.1.2 matplotlib==3.7.1 mdurl==0.1.2 msgpack-python==0.5.6 msgpack-rpc-python==0.4.1 munch==2.5.0 numpy==1.24.2 oauthlib==3.2.2 opencv-contrib-python==4.7.0.72 opencv-python==4.7.0.72 packaging==23.0 pandas==1.5.3 Pillow==9.4.0 pip==23.0.1 protobuf==4.22.1 py-cpuinfo==9.0.0 pyasn1==0.4.8 pyasn1-modules==0.2.8 pyglet==1.5.27 Pygments==2.14.0 pyparsing==3.0.9 python-dateutil==2.8.2 pytz==2022.7.1 PyYAML==6.0 requests==2.28.2 urllib3==1.26.15 wasabi==1.1.1 Werkzeug==2.2.3 wheel==0.38.4 wincertstore==0.2 wrapt==1.15.0 zipp==3.15.0

[ernestum]

Hi @Liuzy0908 to replicate this bug I need to know what environment you used. In the code it says MyEnv, which could be anything. Would you please provide a code snippet that I can execute right away?