search

openai / universe-starter-agent

A starter agent that can solve a number of universe environments.
MIT License
1.1k stars 318 forks source link

non-vnc (first example) pong bot doesn't render with --visualise. #94

Closed NHDaly closed 6 years ago

NHDaly commented 7 years ago

Hi, sorry for so many emails! Thanks for your help! :)

I cannot get the first pong bot (non-vnc) to render, even with the --visualise flag. I'm running macOS 10.12.4 on a 2016 MBP.

When I run the following command, 

python train.py --num-workers 2 --env-id PongDeterministic-v3 --log-dir /tmp/pong --visualise

it does train fine, but it never displays the visualization. I had no problems getting the VNC pong bot to visualize, but not this one. It does open a new instance of the python app in my Dock, but that app has no windows, and doesn't show up in the list of applications when I press cmd+tab.

I verified this works normally: a simple openai gym program renders fine with no problems:

import gym
env = gym.make('PongDeterministic-v3')
while True:
    done = False
    env.reset()
    while not done:
        env.render()
        a = env.action_space.sample()
        _,_,done,_ = env.step(a)

When I run that, I do get the working rendering: screen

But when I run the universe-starter-agent, the window never opens.

To attempt to debug this, I also tried manually adding a call to env.render() in worker.py, as soon as the environment is created (my change here).

This does show a motionless render window for a few seconds, but then it segfaults:

[2017-05-05 19:40:15,360] Events directory: /tmp/pong/train_0
[2017-05-05 19:40:16,111] Starting session. If this hangs, we're mostly likely waiting to connect to the parameter server. One common cause is that the parame
ter server DNS name isn't resolving yet, or is misspecified.
I tensorflow/core/distributed_runtime/master_session.cc:993] Start master session 77b4dab2eb28e027 with config:
device_filters: "/job:ps"
device_filters: "/job:worker/task:0/cpu:0"

[2017-05-05 19:40:18,799] Called reset on <Unvectorize<VectorizeFilter[<class 'envs.DiagnosticsInfoI'>]<AtariRescale42x42<Vectorize<TimeLimit<AtariEnv instanc
e>>>>>> before configuring. Configuring automatically with default arguments
[2017-05-05 19:40:18,801] Starting training at step=0
[2017-05-05 19:40:18,811] Resetting environment
Segmentation fault: 11
(universe-starter-agent) bash-3.2$

What do you think? Do you think there could be something wrong with my setup? I've created the environment following your instructions, and everything else has worked correctly.

Thanks! :)

NHDaly commented 7 years ago

Here's a gist containing the segfault report that Apple reports: https://gist.github.com/NHDaly/1351a18b5c4f2a7837ecfd7786b9e139

And here's some versions of my software:

$ conda list -n universe-starter-agent
# packages in environment at /Users/daly/miniconda3/envs/universe-starter-agent:
#
appdirs                   1.4.3                     <pip>
appnope                   0.1.0                    py35_0
atari-py                  0.0.18                    <pip>
attrs                     16.3.0                    <pip>
autobahn                  17.5.1                    <pip>
Automat                   0.5.0                     <pip>
bleach                    1.5.0                    py35_0
constantly                15.1.0                    <pip>
cycler                    0.10.0                   py35_0
decorator                 4.0.11                   py35_0
docker-py                 1.10.3                    <pip>
docker-pycreds            0.2.1                     <pip>
entrypoints               0.2.2                    py35_1
fastzbarlight             0.0.14                    <pip>
freetype                  2.5.5                         2
go-vncdriver              0.4.19                    <pip>
gym                       0.7.4                     <pip>
hdf5                      1.8.17                        9    conda-forge
html5lib                  0.999                    py35_0
icu                       54.1                          0
incremental               16.10.1                   <pip>
ipykernel                 4.5.2                    py35_0
ipython                   5.2.2                    py35_0
ipython_genutils          0.1.0                    py35_0
ipywidgets                5.2.2                    py35_1
jasper                    1.900.1                       4    conda-forge
jinja2                    2.9.5                    py35_0
jpeg                      9b                            0    conda-forge
jsonschema                2.5.1                    py35_0
jupyter                   1.0.0                    py35_3
jupyter_client            4.4.0                    py35_0
jupyter_console           5.1.0                    py35_0
jupyter_core              4.3.0                    py35_0
libgfortran               3.0.0                         0    conda-forge
libpng                    1.6.27                        0
libtiff                   4.0.6                         7    conda-forge
markupsafe                0.23                     py35_2
matplotlib                2.0.0               np111py35_0
mistune                   0.7.3                    py35_1
mkl                       2017.0.1                      0
mock                      2.0.0                    py35_0    conda-forge
nbconvert                 5.1.1                    py35_0
nbformat                  4.2.0                    py35_0
notebook                  4.3.1                    py35_0
numpy                     1.12.1                    <pip>
numpy                     1.11.3                   py35_0
olefile                   0.44                      <pip>
opencv                    3.1.0               np111py35_1    conda-forge
openssl                   1.0.2k                        0
packaging                 16.8                      <pip>
pandocfilters             1.4.1                    py35_0
path.py                   10.1                     py35_0
pbr                       1.10.0                   py35_0    conda-forge
pexpect                   4.2.1                    py35_0
pickleshare               0.7.4                    py35_0
Pillow                    4.1.1                     <pip>
pip                       9.0.1                    py35_1
prompt_toolkit            1.0.9                    py35_0
protobuf                  3.1.0                    py35_0    conda-forge
ptyprocess                0.5.1                    py35_0
pyglet                    1.2.4                     <pip>
pygments                  2.1.3                    py35_0
PyOpenGL                  3.1.0                     <pip>
pyparsing                 2.1.4                    py35_0
pyparsing                 2.2.0                     <pip>
pyqt                      5.6.0                    py35_2
python                    3.5.2                         0
python-dateutil           2.6.0                    py35_0
pytz                      2016.10                  py35_0
PyYAML                    3.12                      <pip>
pyzmq                     16.0.2                   py35_0
qt                        5.6.2                         0
qtconsole                 4.2.1                    py35_1
readline                  6.2                           2
requests                  2.10.0                    <pip>
scikit-learn              0.18.1              np111py35_1
scipy                     0.18.1              np111py35_1
setuptools                27.2.0                   py35_0
setuptools                35.0.2                    <pip>
simplegeneric             0.8.1                    py35_1
sip                       4.18                     py35_0
six                       1.10.0                   py35_0
sqlite                    3.13.0                        0
tensorflow                0.12.1                   py35_2    conda-forge
terminado                 0.6                      py35_0
testpath                  0.3                      py35_0
tk                        8.5.18                        0
tornado                   4.4.2                    py35_0
traitlets                 4.3.1                    py35_0
Twisted                   17.1.0                    <pip>
txaio                     2.7.1                     <pip>
ujson                     1.35                      <pip>
universe                  0.21.3                    <pip>
wcwidth                   0.1.7                    py35_0
websocket-client          0.40.0                    <pip>
wheel                     0.29.0                   py35_0
widgetsnbextension        1.2.6                    py35_0
xz                        5.2.2                         1
zlib                      1.2.8                         3
zope.interface            4.4.0                     <pip>
$ brew list --versions
cdiff 1.0
cmake 3.7.1 3.7.2 3.8.1
cputhrottle 20100515_1
curl 7.52.1 7.54.0
ffmpeg 3.3
gcc 6.3.0_1
gdb 7.12.1
git 2.11.0 2.11.1 2.12.0 2.12.2
gmp 6.1.2
go 1.8 1.8.1
hexedit 1.2.13
htop 2.0.2
isl 0.18
jpeg-turbo 1.5.1
lame 3.99.5
libdvdcss 1.4.0
libevent 2.0.22 2.1.8
libmpc 1.0.3 1.0.3_1
mono 4.6.2.16 4.8.1.0
mpfr 3.1.5
openssl 1.0.2j 1.0.2k
tmux 2.3_2 2.3_3 2.4
wget 1.18 1.19.1
x264 r2748
xvid 1.3.4
tlbtlbtlb commented 7 years ago

--visualize doesn't work with OSX 10.12, because (unlike 10.11) it requires that OpenGL rendering functions can only be called from the main thread. Universe-starter-agent spawns separate threads to run the environment and train the agent. There may be no easy solution.