search

toybox-rs / Toybox

The Machine Learning Toybox for testing the behavior of autonomous agents.
http://toybox.rs
27 stars 12 forks source link

EpisodicLifeEnv ignores resets when it feels like it #140

Open jjfiv opened 5 years ago

jjfiv commented 5 years ago

Like, when you're telling it to reset it considers whether you actually died and decides to ignore you. I think I've debugged this several times now, thinking there's a bug in our game's reset but there isn't.

anyboby commented 1 year ago

That is intended behavior, it is the exact function of the EpisodicLifeEnv wrapper. See also Machado et al. (2018), "Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents" for more info on this.

jjfiv commented 1 year ago

This behavior was quite annoying when trying to run experiments with the environment. Eg, hard to determine max score achieved by an agent across restarts if sometimes it refuses to restart!

Because this is clearly intended behavior for learning we did not file this issue on any model repos, but only on our experimentation tool, as a note to our future selves.