I want to use the frame stacking technique (4 consecutive frames of images as model input), which works well in PPO-only in SB3.
But after running the above program (about GAIL), the shape of obs collected by rollouts is: (1024,59,256,1). It seems to be collected are single frame images instead of 4 consecutive frames. I think the correct shape of obs should be (1024,59,256,4).
This causes the policy network to fail later when GAIL.train(): The obs input of PPO is (batch, 4, 59, 256), while the obs provided by rollouts is (batch, 1, 59, 526).
How should I solve this problem? Looking forward to your reply.
Hi @Liuzy0908
to replicate this bug I need to know what environment you used. In the code it says MyEnv, which could be anything.
Would you please provide a code snippet that I can execute right away?
Bug description
I want to use the frame stacking technique (4 consecutive frames of images as model input), which works well in PPO-only in SB3.
But after running the above program (about GAIL), the shape of obs collected by rollouts is: (1024,59,256,1). It seems to be collected are single frame images instead of 4 consecutive frames. I think the correct shape of obs should be (1024,59,256,4).
This causes the policy network to fail later when GAIL.train(): The obs input of PPO is (batch, 4, 59, 256), while the obs provided by rollouts is (batch, 1, 59, 526).
How should I solve this problem? Looking forward to your reply.
Steps to reproduce
Environment
pip freeze --all
:absl-py==1.4.0 airsim==1.8.1 cachetools==5.3.0 certifi @ file:///C:/b/abs_85o_6fm0se/croot/certifi_1671487778835/work/certifi chai-sacred==0.8.3 charset-normalizer==3.1.0 cloudpickle==2.2.1 colorama==0.4.6 contourpy==1.0.7 cycler==0.11.0 docopt==0.6.2 filelock==3.9.1 fonttools==4.39.0 gitdb==4.0.10 GitPython==3.1.31 google-auth==2.16.2 google-auth-oauthlib==0.4.6 grad-cam==1.4.6 grpcio==1.51.3 gym==0.21.0 huggingface-hub==0.13.2 huggingface-sb3==2.2.4 idna==3.4 imitation==0.3.2 importlib-metadata==4.13.0 importlib-resources==5.12.0 joblib==1.2.0 jsonpickle==3.0.1 kiwisolver==1.4.4 Markdown==3.4.1 markdown-it-py==2.2.0 MarkupSafe==2.1.2 matplotlib==3.7.1 mdurl==0.1.2 msgpack-python==0.5.6 msgpack-rpc-python==0.4.1 munch==2.5.0 numpy==1.24.2 oauthlib==3.2.2 opencv-contrib-python==4.7.0.72 opencv-python==4.7.0.72 packaging==23.0 pandas==1.5.3 Pillow==9.4.0 pip==23.0.1 protobuf==4.22.1 py-cpuinfo==9.0.0 pyasn1==0.4.8 pyasn1-modules==0.2.8 pyglet==1.5.27 Pygments==2.14.0 pyparsing==3.0.9 python-dateutil==2.8.2 pytz==2022.7.1 PyYAML==6.0 requests==2.28.2 urllib3==1.26.15 wasabi==1.1.1 Werkzeug==2.2.3 wheel==0.38.4 wincertstore==0.2 wrapt==1.15.0 zipp==3.15.0