google-deepmind / open_spiel

OpenSpiel is a collection of environments and algorithms for research in general reinforcement learning and search/planning in games.
Apache License 2.0
4.26k stars 934 forks source link

A question about laser_tag #953

Closed GaoZiHong closed 10 months ago

GaoZiHong commented 2 years ago

Is laser_tag not available for this game? When I run example --game=laser_tag, the following message appears: Creating game..

Starting new game... Initial state: State: ....... ....... ..... ... ..... ....... ....... Orientations: 1 1 Chance Node player -1 sampled outcome: (spawned at location #2) State: ....... ....... ..... ... ..... ....... B...... Orientations: 1 1 Chance Node player -1 sampled outcome: (spawned at location #1) State: ......A ....... ..... ... ..... ....... B...... Orientations: 1 1

player -2 Spiel Fatal Error: InformationStateTensorShape unimplemented. When I run pyspiel.load_game("laser_tag",{"players": 2}) I get the following error: pyspiel.SpielError: Unknown parameter 'players'. Available parameters are: grid, horizon, zero_sum

lanctot commented 2 years ago

Hi @GaoZiHong,

Laser Tag is available. This is due to a bug in the example, which assumes two things: the InformationStateTensor is implemented for all games (which is not true) and that every game has a "players" parameter which is also not true.

If you disable the line accessing the information state tensor and load it with an empty parameter list, it should work.

We will fix soon, thanks for reporting!

GaoZiHong commented 2 years ago

Thank you! i found some other problems. when i run pyspiel.load_game("laser_tag") and use BR as oracle, i get the following errors: File "/home/gaozh/psro-ipdc/psro_ipdc/get_all_states.py", line 75, in _get_subgames_states stop_if_encountered) File "/home/gaozh/psro-ipdc/psro_ipdc/get_all_states.py", line 68, in _get_subgames_states stop_if_encountered) File "/home/gaozh/psro-ipdc/psro_ipdc/get_all_states.py", line 32, in _get_subgames_states is_mean_field = state.current_player() == pyspiel.PlayerId.MEAN_FIELD RecursionError: maximum recursion depth exceeded while calling a Python object

when i use dqn as oracle, i get the following errors: Using 1000 sims per entry. Rectifier : Perturbating oracle outputs : False Sampling from marginals : True Using <function alpharank_strategy at 0x7f1175cc1048> as strategy method. Using <function filter_function_factory..filter_policies at 0x7f1445432d08> as training strategy selector. OpenSpiel exception: /tmp/pip-req-build-tgddg6wg/open_spiel/spiel.cc:779 player >= 0 player = -2, 0 = 0 Traceback (most recent call last): File "/home/gaozh/psro-ipdc/main.py", line 288, in app.run(main) File "/home/gaozh/anaconda3/envs/PSRO/lib/python3.6/site-packages/absl/app.py", line 312, in run _run_main(main, args) File "/home/gaozh/anaconda3/envs/PSRO/lib/python3.6/site-packages/absl/app.py", line 258, in _run_main sys.exit(main(argv)) File "/home/gaozh/psro-ipdc/main.py", line 285, in main gpsro_looper(env, oracle, agents) File "/home/gaozh/psro-ipdc/main.py", line 234, in gpsro_looper symmetric_game=FLAGS.symmetric_game) File "/home/gaozh/psro-ipdc/psro_ipdc/psro.py", line 159, in init **kwargs) File "/home/gaozh/psro-ipdc/psro_ipdc/components/abstract_meta_trainer.py", line 161, in init self._initialize_game_state() File "/home/gaozh/psro-ipdc/psro_ipdc/psro.py", line 173, in _initialize_game_state self.update_empirical_gamestate(seed=None) File "/home/gaozh/psro-ipdc/psro_ipdc/psro.py", line 434, in update_empirical_gamestate self._sims_per_entry) File "/home/gaozh/psro-ipdc/psro_ipdc/components/abstract_meta_trainer.py", line 210, in sample_episodes policies).reshape(-1) File "/home/gaozh/psro-ipdc/psro_ipdc/components/abstract_meta_trainer.py", line 76, in sample_episode return sample_episode(state, policies) File "/home/gaozh/psro-ipdc/psro_ipdc/components/abstract_meta_trainer.py", line 76, in sample_episode return sample_episode(state, policies) File "/home/gaozh/psro-ipdc/psro_ipdc/components/abstract_meta_trainer.py", line 62, in sample_episode state_policy = policies[player](state, player) File "/home/gaozh/psro-ipdc/psro_ipdc/policy.py", line 141, in call return self.action_probabilities(state, player_id) File "/home/gaozh/psro-ipdc/psro_ipdc/components/rl_policy.py", line 67, in action_probabilities state.information_state_tensor(cur_player)) pyspiel.SpielError: /tmp/pip-req-build-tgddg6wg/open_spiel/spiel.cc:779 player >= 0 player = -2, 0 = 0

i found that in the game laser_tag, the state is different with other game, for example, the state.current_player() cannot return correct value. how could i solve these problems? what's the correct way to use laser_tag game?

lanctot commented 2 years ago

Hi @GaoZiHong,

Laser Tag is a simultanous move game. So it will return the kSimultaneousPlayerId (== -2) for current_player() at decisions nodes.

So my guess is that you're using it in a context that assumes the turn-based setting. You can translate any simultaneous game into a turn-based one through a wrapper. Here is an example of how to do that:

pyspiel.load_game("turn_based_simultaneous_game(game=blotto(players=3,coins=5))")

In your case:

pyspiel.load_game("turn_based_simultaneous_game(game=laser_tag())")

I recommend taking a look at the OpenSpiel paper section 3 to read more about how games are formalized in OpenSpiel (specifically how simultaneous-move games compare to turn-based extensive-form games).

GaoZiHong commented 2 years ago

Thank you very much!!! I run the code, pyspiel.load_game("turn_based_simultaneous_game(game=laser_tag())"), and make the following changes: In the file rl_policy.py, information_state_tensor should be replaced by observation_tensor. The maximum recursion depth should be set to a larger value, such as 5000, to avoid the problem of exceeding the recursion depth when executing sample_episode. Thanks again for your great contribution!

lanctot commented 10 months ago

We will fix soon, thanks for reporting!

Ok.. wasn't "soon" but better late than never? https://github.com/google-deepmind/open_spiel/pull/1161 :)