search

openai / gym

A toolkit for developing and comparing reinforcement learning algorithms.
https://www.gymlibrary.dev
Other
34.66k stars 8.6k forks source link

Bug as score reachs 1 million #978

Closed sergio-hcsoft closed 5 years ago

sergio-hcsoft commented 6 years ago

When some games reach a score of 999,999 and get a +1 point, they reset to score 0, both on screen and internally, so it is not possible to reach some human records from the gym.

Games affected by the "1M bug" are, at least: "Chopper Command", "Daemon Attack", "Frostbite", "Gopher", "Seaquest", "Up N down" and "Video Pinball" (you can watch the AI avoiding making new points after reaching 999,999 here: https://www.youtube.com/watch?v=ciSRCehM10E).

Not all games are affected, you can reach 2M in Asteroids and Centipede or even +10M in Atlantis, so it is not a limitation of the Atari-2600 console but of some of the original games.

Those scores were obtained using https://github.com/FragileTheory/FractalAI and, more specifically, the "Swarm_Wave_example.ipynb" (there is a one-core version on the master branch, and a faster (but alpha) version on the "parallel" branch).

christopherhesse commented 5 years ago

This seems like an issue with ALE/atari_py, is that correct?

christopherhesse commented 5 years ago

Closing due to lack of activity.