Help wanted: single player mode

[lukaszkn]

Hi @enpasos,

Any idea how to implement single player games? E.g. Cartpole, Lunar Lander or Breakout? And print out current max score archived.

Thanks in advance.

[enpasos]

Hi @lukaszkn.

So far I've been focusing on board games, with the goal of getting MuZero running stably up to Go9x9, with proof of the level of play achieved. I felt that this was necessary for me to understand how to get MuZero running successfully on a small budget. Based on this, I would like to redesign the code so that each game can be easily plugged into the general framework, with clear separation of the framework and the respective game with its environment.

What needs to be done for Atari like games:

• "fast thinking" - neural network: add the reward network, adapt the presentation network.
• makeTarget method in training: there should be a configuration switch when to use a reward network and when not. Depending on this switch, the target value contains the final reward (without reward network) or not (with reward network).
• "Slow thinking" - mcts: use the reward configuration switch - I think only one place - besides that it should word as is, but bugs need to be detected and removed.
• make symmetry groups configurable - is currently hardcoded, but is a game-specific feature
• find a Java friendly implementation (or mix with Python)
• "maximum score" as test result

[enpasos]

For single player games with only a final reward you certainly need less - something like:

• make symmetry groups configurable - is currently hardcoded, but is a game-specific feature
• find a Java friendly environment implementation
• "maximum score" as test result
• single player mode switch for mcts

[enpasos]

Added:

• single player mode (still running with final reward only - therefore still no reward network)
• example Peg Solitair, for illustration after some training ...

[lukaszkn]

Nice, thanks much for your work!