Neonrace indefinitely using only ArrowUp

[MarioBross69]

Hi,

Testing Neonrace with four workers, I note that at the beginning of the game all the keys are used, ArrowUp, left and right arrow, but after a few hours only the ArrowUp key is used, and no longer use right and left arrow, therefore it has never just passed the first level, always gets off the road, even after 48 hours stays indefinitely using only ArrowUp, however in the documentation it says that:

Getting 80% of the maximal score takes between 1 and 2 hours with 16 workers, and getting to 100% of the score takes about 12 hours

In my case never goes beyond the first level and of course never reaches 100%

Is this behavior normal?

Thanks

[tlbtlbtlb]

It's not normal. It can happen if your computer isn't fast enough to run 4 workers + 4 environments. You should have at least 8, preferably 12 cores to run that many.

In the output of the workers, look for reaction_time=XX.XXmS. If it's higher than 150 mS, you'll get bad results. Either use fewer workers or run on a machine with more cores.

[MarioBross69]

I think my reaction time is correct

Sending out new action probe: [('KeyEvent', 96, True), ('KeyEvent', 96, False)] Next probe received: old=1483654095.98 new=1483654096.08 Sending out new action probe: [('KeyEvent', 96, True), ('KeyEvent', 96, False)]

Stats for the past 5.04s: vnc_updates_ps=36.3 n=1 reaction_time=69.03ms observation_lag=31.51ms+-1.91ms action_lag=10.59ms+- 2.15ms processing_lag=1.53ms+-0.28ms thinking_lag=25.41ms+-1.64ms reward_ps=0.0 reward_total=0.0 vnc_bytes_ps[total]=378679.9 vnc_pixels_ps[total]=382 1775.7 reward_lag=None rewarder_message_lag=76.10ms+-0.00ms fps=4 [0:localhost:5901] RewardState: popping reward 0.0 from episode_id 2 Next probe received: old=1483654096.08 new=1483654096.17

Sending out new action probe: [('KeyEvent', 96, True), ('KeyEvent', 96, False)] Next probe received: old=1483654096.17 new=1483654096.35 Sending out new action probe: [('KeyEvent', 96, True), ('KeyEvent', 96, False)]

Anyway I will try in a machine with more cores

Thanks for your help

[tlbtlbtlb]

I changed the default action space of NeonRace, to allow simultaneous keys. Previously, it would only use 1 of the 5 keys at a time. Now it can also use up left and up right and up x combinations, which human players use.

Code at https://github.com/openai/universe-starter-agent/blob/master/envs.py#L44

[MarioBross69]

Maybe it's my installation fault but after this universe-starter-agent commits, neonrace has stopped working, no key is pressed, the car remains unmoved

Should I upgrade also gym and universe?

[tlbtlbtlb]

Probably better to upgrade. Can you include log files from a run that doesn't work?

[MarioBross69]

In this test goes into a loop and does not get to start the race

This only happens to me since I updated universe-starter-agent, before it worked correctly I have not updated neither gym nor universe

This is my call command: Python train.py --num-workers 4 --env-id flashgames.NeonRace-v0 --log-dir / tmp / neonrace

I have modified the port of tensorflow to 6006 in train.py, but this I do not think it matters

universe-8403_log.txt

[tlbtlbtlb]

Near the top of universe-8403_log.txt:

[2017-01-09 22:33:30,920] [0] Creating container: image=quay.io/openai/universe.flashgames:0.20.7. Run the same thing by hand as: docker run -p 5900:5900 -p 15900:15900 --cap-add SYS_ADMIN --ipc host --privileged quay.io/openai/universe.flashgames:0.20.7
[2017-01-09 22:33:31,342] [0] Could not start container: 500 Server Error: Internal Server Error ("{"message":"driver failed programming external connectivity on endpoint universe-w3SZnK-0 (127c49837154ab34b05d38e743f5ad34771547515dae5bcb2af5eccd83b79d7d): Bind for 0.0.0.0:15900 failed: port is already allocated"}")

Most likely there is already a docker container running with the same port. docker kill $(docker ps -q) will kill all containers so you can start fresh.

You should also upgrade Universe -- it has better handling of these errors now.

[MarioBross69]

Neither "docker kill nor reboot" system solves the problem, Neonrace is still in an infinite loop, just when it will start the race returns to the initial menu Tomorrow I will update universe to see if the problem is solved Thank you

[MarioBross69]

Confirmed!, updating gym and universe works correctly again Thanks for your help

[crci]

@MarioBross69 did you ever reached a state in which the car actually turned? If so how did you do it? I am also observing this behaviour of the agent just pressing Arrow up, it seems perhaps some sort of local optima.