search

Farama-Foundation / Arcade-Learning-Environment

The Arcade Learning Environment (ALE) -- a platform for AI research.
https://ale.farama.org/
GNU General Public License v2.0
2.14k stars 420 forks source link

RLGlue Controller Issues #41

Closed mhauskn closed 9 years ago

mhauskn commented 9 years ago

Two issues with the RLGlueController: 1) There is no way to get legal actions. When the -use_restricted_actions flag is true, then it will return the restricted actions for player A only. When false, it returns all 18 actions for both player and player b. 2) Non-uniform handling of one-player/two-player. By default, action_spec="ACTIONS INTS (0 17)(18 35)". With -use_restricted_actions, action_spec="ACTIONS INTS (0 Minimal_action_set_size)".

Solution to problem 1: By default set action_spec="ACTIONS INTS (0 legal_action_set_size)". However do we want to include player B's actions?

Possible solutions to problem 2: 1) Convert everything in the RLGlue interface to either single or multiplayer. 2) Add a rl-glue specific flag to specify if you want single vs multiplayer and return the corresponding single or multiplayer action set.

mcmachado commented 9 years ago

I don't think anyone is concerned with multiplayer in Atari right now.

2015-02-22 15:06 GMT-07:00 Matthew Hausknecht notifications@github.com:

Two issues with the RLGlueController: 1) There is no way to get legal actions. When the -use_restricted_actions flag is true, then it will return the restricted actions for player A only. When false, it returns all 18 actions for both player and player b. 2) Non-uniform handling of one-player/two-player. By default, action_spec="ACTIONS INTS (0 17)(18 35)". With -use_restricted_actions, action_spec="ACTIONS INTS (0 Minimal_action_set_size)".

Solution to problem 1: By default set action_spec="ACTIONS INTS (0 legal_action_set_size)". However do we want to include player B's actions?

Possible solutions to problem 2: 1) Convert everything in the RLGlue interface to either single or multiplayer. 2) Add a rl-glue specific flag to specify if you want single vs multiplayer and return the corresponding single or multiplayer action set.

— Reply to this email directly or view it on GitHub https://github.com/mgbellemare/Arcade-Learning-Environment/issues/41.

Marlos Cholodovskis Machado http://www.dcc.ufmg.br/~marlos http://www.cs.ualberta.ca/~machado

mhauskn commented 9 years ago

I agree, however the Stella Environment supports it, so it seems like a shame to not to support it in the RLGlueInterface.

However, RLGlue says it's only a single agent learning platform (http://glue.rl-community.org/wiki/RL-Glue_Core#Does_RL-Glue_support_multi-agent_reinforcement_learning.3F) so if RLGlue itself denies multi-agent support, maybe we follow their lead.

mgbellemare commented 9 years ago

Yes, I would also go this way. It's probably best to keep it single-player.

Let's make sure to document all the interface changes in the next release.

mhauskn commented 9 years ago

This was addressed in commit #42. RLGlue interface was refactored to be single player. Closing issue.