custom gym environments are not compatible with current gym API

[MatthewCaseres]

I think you are depending on an older version of gym in some of the environments you made, a common error is

/usr/local/lib/python3.9/site-packages/gym/wrappers/time_limit.py in step(self, action)
     48 
     49         """
---> 50         observation, reward, terminated, truncated, info = self.env.step(action)
     51         self._elapsed_steps += 1
     52 

ValueError: not enough values to unpack (expected 5, got 4)

walk_env.py has step function return return (int(s), r, d, {"prob": p}) but the version of gym I'm on (0.26.0) seems to wrap your environment in some time limit and gym.make('RandomWalk-v0') returns something like this <TimeLimit<OrderEnforcing<PassiveEnvChecker<WalkEnv<RandomWalk-v0>>>>>.

Your dockerfile doesn't pin gym to any specific version so I don't know what the right version is.

[MatthewCaseres]

here is a minimal reproduction - https://github.com/MatthewCaseres/gdrl_reproduction/tree/main

[mimoralea]

Can you pull the image instead of building it from scratch and check the gym version? I'm aware they have changed the gym package considerably in recent months. I will be updating the codebase with the second edition of the book.

[MatthewCaseres]

I feel that this issue should not have been closed so quickly? I assumed that since no version of gym is specified that it would default to the latest? https://github.com/mimoralea/gdrl/blob/master/docker/Dockerfile#L240

[mimoralea]

Correct. Let me try pushing the right versions soon.

[mimoralea]

Package                Version            
---------------------- -------------------
alembic                1.3.2              
asciinema              2.0.2              
async-generator        1.10               
atari-py               0.2.6              
attrs                  19.3.0             
backcall               0.1.0              
beautifulsoup4         4.8.2              
bleach                 3.1.0              
blinker                1.4                
bokeh                  1.3.4              
Box2D                  2.3.2              
certifi                2019.11.28         
certipy                0.1.3              
cffi                   1.13.2             
chardet                3.0.4              
Click                  7.0                
cloudpickle            1.2.2              
conda                  4.7.12             
conda-package-handling 1.6.0              
cryptography           2.8                
cycler                 0.10.0             
Cython                 0.29.14            
cytoolz                0.10.1             
dask                   2.3.0              
decorator              4.4.1              
defusedxml             0.6.0              
dill                   0.3.1.1            
distributed            2.5.2              
entrypoints            0.3                
fastcache              1.1.0              
fsspec                 0.6.2              
future                 0.18.2             
gmpy2                  2.1.0b1            
gym                    0.15.4             
gym-aima               0.0.1              
gym-bandits            0.1                
gym-walk               0.0.1              
h5py                   2.9.0              
HeapDict               1.0.1              
idna                   2.8                
imageio                2.6.1              
importlib-metadata     1.3.0              
ipykernel              5.1.3              
ipython                7.11.0             
ipython-genutils       0.2.0              
ipywidgets             7.5.1              
jedi                   0.15.2             
Jinja2                 2.10.3             
joblib                 0.14.1             
json5                  0.8.5              
jsonschema             3.2.0              
jupyter-client         5.3.3              
jupyter-core           4.6.1              
jupyterhub             1.0.0              
jupyterlab             1.2.1              
jupyterlab-server      1.0.6              
kiwisolver             1.1.0              
llvmlite               0.29.0             
locket                 0.2.0              
Mako                   1.1.0              
MarkupSafe             1.1.1              
matplotlib             3.1.2              
mistune                0.8.4              
more-itertools         8.0.2              
mpmath                 1.1.0              
msgpack                0.6.2              
nbconvert              5.6.1              
nbformat               4.4.0              
networkx               2.4                
notebook               6.0.0              
numba                  0.45.1             
numexpr                2.6.9              
numpy                  1.17.3             
oauthlib               3.0.1              
olefile                0.46               
opencv-python          4.1.2.30           
packaging              19.2               
pamela                 1.0.0              
pandas                 0.25.3             
pandocfilters          1.4.2              
parso                  0.5.2              
partd                  1.1.0              
patsy                  0.5.1              
pexpect                4.7.0              
pickleshare            0.7.5              
Pillow                 6.2.1              
pip                    19.3.1             
prometheus-client      0.7.1              
prompt-toolkit         3.0.2              
protobuf               3.9.2              
psutil                 5.6.7              
ptyprocess             0.6.0              
pybullet               2.6.3              
pycosat                0.6.3              
pycparser              2.19               
pycurl                 7.43.0.3           
pyglet                 1.3.2              
Pygments               2.5.2              
PyJWT                  1.7.1              
pyOpenSSL              19.1.0             
pyparsing              2.4.6              
pyrsistent             0.15.6             
PySocks                1.7.1              
python-dateutil        2.8.1              
python-editor          1.0.4              
pytz                   2019.3             
PyWavelets             1.1.1              
PyYAML                 5.2                
pyzmq                  18.1.1             
requests               2.22.0             
ruamel-yaml            0.15.80            
scikit-image           0.15.0             
scikit-learn           0.21.3             
scipy                  1.4.1              
seaborn                0.9.0              
Send2Trash             1.5.0              
setuptools             42.0.2.post20191201
six                    1.13.0             
sortedcontainers       2.1.0              
soupsieve              1.9.4              
SQLAlchemy             1.3.11             
statsmodels            0.10.2             
sympy                  1.4                
tabulate               0.8.6              
tblib                  1.6.0              
terminado              0.8.3              
testpath               0.4.4              
toolz                  0.10.0             
torch                  1.3.1              
torchvision            0.4.2              
tornado                5.1.1              
tqdm                   4.41.0             
traitlets              4.3.3              
typing                 3.6.4              
urllib3                1.25.7             
vincent                0.4.4              
wcwidth                0.1.7              
webencodings           0.5.1              
wheel                  0.33.6             
widgetsnbextension     3.5.1              
xlrd                   1.2.0              
zict                   1.0.0              
zipp                   0.6.0