Closed spascience closed 2 years ago
tictactoe_v3.env().possible_agents
is ['player_1', 'player_2']
. Changing it to:
"policies": { "player_1": (None, obs_space, act_space, {}), "player_2": (None, obs_space, act_space, {}), },
seems to work.
P.s. for future reference, you can check the actual error raised by RLlib in e.g. /.../ray_results/DQN/DQN_tictactoe_9dc71_00000_0_2022-07-24_13-46-59/error.txt
.
@spascience I think Rohan has made a PR fixing these tutorials if you're interested, it'll probably be a couple days before they're merged.
(Just to clarify, my PR is unrelated to this issue-the PR is RLlib updates, the bug was just incorrect agent ids)
@Rohan138 Ah ok, my bad.
Thanks! Changing line ~96 as you described worked for me.
bug description Using Ray[rllib] 1.13.0 and pettingzoo 1.19.0, I'm having difficulty training DQN on the TicTacToe env. The simplest reproduction I've found is
rllib_leduc_holdem.py
, which runs fine; however, it will error when the environment is switched to TicTacToe:I'm new to both PettingZoo and RLlib, so I'm happy to hear advice on how to move forward. Is there any relevant difference in the TicTacToe and Leduc Hold`em environments for training with RLlib? Any reason we shouldn't use the same script?
Code example The following script is copied from the latest release's holdem with rllib tutorial. It has minor edits (see comments with "edit") to switch the environment to
tictactoe_v3
and update some import names:Log and truncated traceback (apologies for the long section, scroll to end for error):
and later:
System Info
The latest ray only supports up to gym 0.24 and pettingzoo actually wants gym 0.24.1 or later (only change between 1.19 and latest 1.19.1 is updating the gym requirement). I'm not sure if it should matter in this case, since the script runs fine for me on leduc_holdem.
Additional context Note that TicTacToe and Holdem are not positive sum games (which is a soft "requirement" of RLlib's PettingZooEnv wrapper). But this shouldn't matter for just getting something to run.
Checklist
There are several (+1 year old) discussions on RLlib/pettingzoo interfacing. Most recently there is a motion to adopt pettingzoo's api into Ray-RLlib's multi agent setting (see https://github.com/ray-project/ray/issues/23975#issue-1207213022), which I'm for. Obviously this issue would be impacted.