search

openai / universe-starter-agent

A starter agent that can solve a number of universe environments.
MIT License
1.1k stars 318 forks source link

Installation instructions don't work #133

Closed AdamStelmaszczyk closed 6 years ago

AdamStelmaszczyk commented 6 years ago

I followed install instructions from the README.

After running python train.py --num-workers 2 --env-id PongDeterministic-v3 --log-dir /tmp/pong --visualise --mode child and tail -f /tmp/pong/*.out I got:

==> /tmp/pong/a3c.w-0.out <==
RuntimeError: module compiled against API version 0xb but this version of numpy is 0xa
Traceback (most recent call last):
  File "worker.py", line 3, in <module>
    import go_vncdriver
  File "/home/adam/miniconda3/envs/universe-starter-agent/lib/python3.5/site-packages/go_vncdriver/__init__.py", line 9, in <module>
    from go_vncdriver.go_vncdriver import *
ImportError: numpy.core.multiarray failed to import

conda list shows 2 versions of numpy installed. First, numpy==1.13.3 is installed as a dependency for pip install "gym[atari]". But then numpy==1.11.3 is installed because of conda install -y -c https://conda.binstar.org/menpo opencv3.

I removed the numpy==1.13.3 installed by pip. But the error ImportError: numpy.core.multiarray failed to import stays.

So, I installed numpy==1.13.3 with pip again, but removed numpy==1.11.3 coming from conda:

$ conda uninstall numpy
Fetching package metadata ...........
Solving package specifications: .

Package plan for package removal in environment /home/adam/miniconda3/envs/universe-starter-agent:

The following packages will be REMOVED:

    numpy:   1.11.3-py35_0                                     
    opencv3: 3.2.0-np111py35_0  https://conda.binstar.org/menpo
    scipy:   0.19.0-np111py35_0                                

Proceed ([y]/n)? y

It also removes opencv3 and scipy. However, I installed again the same versions:

pip install opencv-python==3.2.0.8
pip install scipy==0.19.0

Then after running train.py I got:

    sess.run(init_op, feed_dict=init_feed_dict)
  File "/home/adam/miniconda3/envs/universe-starter-agent/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 889, in run
    run_metadata_ptr)
  File "/home/adam/miniconda3/envs/universe-starter-agent/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1120, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/adam/miniconda3/envs/universe-starter-agent/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run
    options, run_metadata)
  File "/home/adam/miniconda3/envs/universe-starter-agent/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Op type not registered 'VariableV2'

Tensorflow version installed with pip install tensorflow was 1.4.1. I noticed that in Dependencies you point to 0.12. I removed 1.4.1 and did pip install tensorflow==0.12.

I ran train.py again.

This time the console logs seems ok, but the visualization looks wrong:

pong

I tried other games from Atari and the visualization looks wrong in the same way. I don't know if this is only visualization or visualization + network input (so the whole training may be wrong).

How to make it work?

$ uname -a
Linux adam-ThinkPad-T520 4.8.0-53-generic #56~16.04.1-Ubuntu SMP Tue May 16 01:18:56 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
$ conda list
atari-py                  0.1.1                     <pip>
attrs                     17.3.0                    <pip>
autobahn                  17.10.1                   <pip>
Automat                   0.6.0                     <pip>
bleach                    1.5.0                     <pip>
ca-certificates           2017.08.26           h1d4fec5_0  
cairo                     1.14.10              hdf128ce_6  
certifi                   2017.11.5        py35h9749603_0  
constantly                15.1.0                    <pip>
docker-py                 1.10.3                    <pip>
docker-pycreds            0.2.1                     <pip>
enum34                    1.1.6                     <pip>
fastzbarlight             0.0.14                    <pip>
fontconfig                2.12.4               h88586e7_1  
freetype                  2.8                  hab7d2ae_1  
future                    0.16.0                    <pip>
glib                      2.53.6               h5d9569c_2  
go-vncdriver              0.4.19                    <pip>
gym                       0.7.4                     <pip>
harfbuzz                  0.9.39                        1  
hdf5                      1.8.17                        2  
html5lib                  0.9999999                 <pip>
hyperlink                 17.3.1                    <pip>
icu                       58.2                 h9c2bf20_1  
idna                      2.6                       <pip>
incremental               17.5.0                    <pip>
intel-openmp              2018.0.0             hc7b2577_8  
jpeg                      9b                   h024ee3a_2  
libedit                   3.1                  heed3624_0  
libffi                    3.2.1                hd88cf55_4  
libgcc-ng                 7.2.0                h7cc24e2_2  
libgfortran               3.0.0                         1  
libpng                    1.6.32               hbd3595f_4  
libstdcxx-ng              7.2.0                h7a57d05_2  
libtiff                   4.0.9                h28f6b97_0  
libxcb                    1.12                 hcd93eb1_4  
libxml2                   2.9.4                h2e8b1d7_6  
Markdown                  2.6.10                    <pip>
mkl                       2017.0.4             h4c4d0af_0  
ncurses                   6.0                  h9df7e31_2  
numpy                     1.13.3                    <pip>
opencv-python             3.2.0.8                   <pip>
openssl                   1.0.2n               hb7f436b_0  
pcre                      8.41                 hc27e229_1  
Pillow                    4.3.0                     <pip>
pip                       9.0.1            py35h7e7da9d_4  
pixman                    0.34.0               hceecf20_3  
protobuf                  3.1.0                     <pip>
pyglet                    1.3.0                     <pip>
PyOpenGL                  3.1.0                     <pip>
python                    3.5.4               h417fded_24  
PyYAML                    3.12                      <pip>
readline                  7.0                  ha6073c6_4  
requests                  2.10.0                    <pip>
scipy                     0.19.0                    <pip>
setuptools                36.5.0           py35ha8c1747_0  
six                       1.11.0                    <pip>
sqlite                    3.20.1               hb898158_2  
tensorflow                0.12.0                    <pip>
tensorflow-tensorboard    0.4.0rc3                  <pip>
tk                        8.6.7                hc745277_3  
Twisted                   17.9.0                    <pip>
txaio                     2.8.2                     <pip>
ujson                     1.35                      <pip>
universe                  0.21.3                    <pip>
urllib3                   1.22                      <pip>
websocket-client          0.44.0                    <pip>
Werkzeug                  0.13                      <pip>
wheel                     0.30.0           py35hd3883cf_1  
xz                        5.2.3                h55aa19d_2  
zlib                      1.2.11               ha838bed_2  
zope.interface            4.4.3                     <pip>
AdamStelmaszczyk commented 6 years ago

After hours of debugging, I found it...

The culprit is pip install universe.

It installs a bit old universe 0.21.3 as this is the latest on pypi.

If you install the HEAD version from the universe git repo (pip install git+https://github.com/openai/universe) the visualizations look good.

It works with Python 2.7, it hangs with Python 3.5, 3.6.

I bisected top commits there to see which one was fixing the problem for Python 2.7, this one: https://github.com/openai/universe/commit/a4f78eb928ef2dc26247b16a75a4e862c2552f00

It bumps the version to 0.21.5, however this version is not on pypi. Both universe-starter-agent and universe repos look unmaintained, sad.

I'm not closing the issue as in my eyes issues should be only closed when the problem/source was fixed, here it's not the case, it won't work for newcomers executing installation instructions from README. Hopefully they will see this issue page. Feel free to close when fixed.