openai / gym

A toolkit for developing and comparing reinforcement learning algorithms.
https://www.gymlibrary.dev
Other
34.74k stars 8.61k forks source link

Some atari videos are rendered with flashing black screen #280

Closed ppwwyyxx closed 5 years ago

ppwwyyxx commented 8 years ago

For example, here: https://gym.openai.com/evaluations/eval_zIeNk5MxSGOmvGEUxrZDUw I observed the same thing when I used monitor to record a video.

Haven't checked if the observed state is also like this.

lakshayg commented 7 years ago

A solution to this problem was pointed out in #378 by @tlbtlbtlb. I am quoting his comment here

There are versions of the environments that do what I think you want, formed by adding Deterministic or NoFrameSkip to the name. For example, env.make("AirRaidDeterministic-v3").

You can see a list of all the available environments at https://pastebin.com/pefCGyL7 or by running the script https://github.com/openai/gym/blob/master/examples/scripts/list_envs

doronsobol commented 7 years ago

I tried @lakshayg idea and ran all the versions of "AirRaid" and I still get mostly black screen.

Using older versions of gym I don't get black screens but those weird images: image instead of: image

If someone has an idea of how to fix that, either by fixing the images of the older version, or by other methods it would be very helpfull.

the code I used: env = gym.make('AirRaidNoFrameskip-v0') env.reset() game_count = 1 for i in range(50000): action_n = env.actionspace.sample() state, , done, info = env.step(action_n) env.render() if done: env.reset()

Thanks

tlbtlbtlb commented 7 years ago

You probably have incompatible versions of gym and atari-py. Try pip install atari-py.

doronsobol commented 7 years ago

@tlbtlbtlb Thanks for the quick answer, but I'm using the latest atari-py (0.1.1) and gym (0.9.4) and I keep getting black screens - specifically in this game (AirRaid) in all of it's versions.

tlbtlbtlb commented 7 years ago

Random intervals of blackness are part of the AirRaid game. I think it's simulating the city shutting off the power so the bombers can't see it clearly. https://en.wikipedia.org/wiki/Air_Raid_(video_game).

It's not a great game, and it was a commercial failure back in the 70s (only a few were sold.) It does seem like an interesting one for RL research, because the agent needs to remember where the attackers are when the screen is black.

A good sanity check on the game-vs-gym behavior is to install Stella which lets you play all the Atari games. It's called "AIR_RAID.BIN" in the menu.

doronsobol commented 7 years ago

I played this game in Stella and didn't get those black frames, I seems that in the gym almost all of the frames are black.

tlbtlbtlb commented 7 years ago

Playing under Stella 4.7.3, most of the frames are black. Screen capture from my Mac: https://www.youtube.com/watch?v=tuMfNpsNSns&feature=youtu.be

ppwwyyxx commented 7 years ago

That's weird. I was playing it with stella 5.0.2 on Linux. Can't see those black frames. Rom downloaded from atari-py: https://github.com/openai/atari-py/blob/master/atari_py/atari_roms/air_raid.bin

wookayin commented 6 years ago

For those who might find this thread in the future: see also #596.

trav1s commented 6 years ago

I had the same issue as @doronsobol with other atari games like breakout and pong. its clearly a display bug and not normal. The workaround from @stefanutti also worked for me: pip install atari-py==0.0.21

seems some kind of version incompatibility between atari-py and other packages.

wookayin commented 6 years ago

Any solutions with using recent version of atari-py, say 0.1.6? Why is it happening?

christopherhesse commented 5 years ago

I looked into the original air_raid issue here: https://github.com/openai/gym/issues/1650 and it looks like it's a problem with ALE.