Getting 7 as action for a game with 3 actions

[arjung128]

I have been trying to train an online agent on the environment FreewayNoFrameskip-v4. Because this gym environment is not deterministic, I seeded the environment. Specifically, in atari_lib.py, I added

• env.seed(0) after env = gym.make(full_game_name) in create_atari_environment
• self.environment.seed(0) at the end of the AtariPreprocessing class's __init__ function
• self.environment.seed(0) at the start of the reset function in the AtariPreprocessing class

No other changes were made. I then used the Batch RL codebase to train an online agent.

In all of training, there was one instance of a 7 stored as the action (specifically the last action at the end of five iterations of training), even though Freeway only has three actions. All other stored actions were {0, 1, 2}. Any ideas what could be the cause of this? Going in and changing this one 7 to the most common action isn't a problem, but if this problem arises repeatedly, and for other games, it could be difficult to deal with.

[psc-g]

hi, is this still an issue? are you sure you're reloading the correct checkpoint (for the same game)? otherwise, it seems like it might make more sense to ask in the batch rl repo?