Python Bots

[nicolodavis]

General plan to implement a Python client that people can use to write bots. The client will just use a websocket implementation to communicate with the Node server (there should be no changes server-side):

• [ ] Investigate Python websocket implementation.
• [ ] API for Python bots.
• [ ] Python bot for Tic-Tac-Toe.

[akaguny]

why pyton?)

[nicolodavis]

Python is an important language for the AI community and will open up many more possibilities.

[vdfdev]

To make things simpler, we could support some kind of communication between node <---> python, so we would not need to support websockets on python, or change our implementation. We just need to flag this bot as "server only", to avoid trying to run it on the browser.

[francoijs]

Hey, I had the same need for a python bot to replace one of the players. You can find a prototype here: https://github.com/francoijs/TicTacBot The bot plays TicTacToe (randomly) against the player: state and actions are exchanged with a python subprocess (using stdin/stdout). The main problem for the moment is that the bot has trouble emitting 'endTurn' and 'endPhase' events (it works somehow in the Tic-tac-toe example, but I never managed to get 'endPhase' events to the server with a more complex game) I would love to have your feedback/suggestions!

[nicolodavis]

@francoijs This looks like a good first step in this direction!

Here are my thoughts:

It would be nice to have a Python API that abstracts away how the bot communicates with the server (and resembles the JS API). For example:

from boardgame.io import Bot

class TicTacToeBot(Bot):

  # Called when it is this bot's turn to play.
  def think(G, ctx):
    self.moves.clickCell(3)
    self.events.endTurn()

I personally think that it's much less messy if the bot just uses websockets behind the scenes and communicates with the server like any other JS client. This will allow the server code to remain untouched and you don't have to deal with stdin / stdout parsing.

[francoijs]

Thanks for the feedback, your design idea is indeed much cleaner. I will work toward adapting my prototype and get back to you.

[francoijs]

The prototype was updated: it now connects to the server using the Socket.IO python client and can emit actions until game is over. Next: emitting 'endTurn' and 'endPhase' events to support more complex games.
Example of instantiation:

from boardgameio import Bot

class TicTacToeBot(Bot):

    def __init__(self):
        Bot.__init__(self, server='localhost', port=8000, game_name='default', num_players=2, player_id='1')

    # Called when it is this bot's turn to play.
    def think(self, G, ctx):
        cells = G['cells']
        idx = -1
        while True and None in cells:
            idx = random.randint(0, len(cells)-1)
            if not cells[idx]:
                break
        return self.make_move('clickCell', idx)

    # Called when game is over.
    def gameover(self, G, ctx):
        print 'winner is '+ctx['gameover']

[nicolodavis]

This is quite exciting @francoijs! Will take a look in the next couple of days.

[nicolodavis]

Looks good @francoijs! Do you want to send me a PR to merge it in this repo? Just keep it in a directory called python at the root level.

One comment: You should look at actionPlayers and not currentPlayer to determine if it is the bot's turn to play.

[francoijs]

Here it is: #195

[nicolodavis]

I just realized that for bots to be useful, they need some mechanism to simulate future moves. Do you have any thoughts on how we might do this for Python bots, given that the game engine is in JavaScript?

[jstacoder]

If you need to call functions from the engine in python you can use PyExecJs, which gives you access to the js runtime, so you can call JavaScript functions and return the results to the calling python code.

[francoijs]

@jstacoder: thanks for the hint, that could prove useful! @nicolodavis: By 'simulating future moves', do you mean implementing the game logic to process the transition from one game state G to another, given any action (just like in the methods Game.moves.*)? Would it mean providing an equivalent to the base class 'Game' in python?

My initial need for the python bot was a mean of connecting an external python implementation of a game logic to the boardgame.io implementation. (I am using the python bot to connect a DQN-based AI that I had already trained independently in pure python).

[nicolodavis]

@francoijs For the short term, I think what you've created will serve us quite well. People can write bots in JS using the API here, and they can use Python for the reinforcement learning stuff that they train offline.

I don't think we should implement any game logic in Python (that would essentially be a separate framework). I was hoping that we could create some sort of bridge that allows executing the game logic in JS but called from Python (either by using PyExecJs or bringing up a Node server that exposes an API to return the result of applying a game move on a give state). That way, people can also write MCTS bots in Python with a similar API that we provide for JS.

[nicolodavis]

https://github.com/PiotrDabkowski/Js2Py also looks promising if we want to leverage existing game logic written in JS in a Python bot.

[francoijs]

About the concept of making available the JS game logic through a python API:

• I've implemented an OpenAI 'Gym' environment around the game.js from Tic-Tac-Toe, using the Js2Py library
• the biggest problem is that the translated JS code is very very slow (1.5s to play 1 episode or 9 turns of TicTacToe, where it takes 0.5 ms in pure python on the same machine) So basically it's useless from an AI point of view, where you need to run thousands of games. (for reference on how to use js2py for this, the code is here)

[nicolodavis]

Thanks for looking into this @francoijs!

Yes, the downside of a non-native implementation is that you incur these performance costs. I think the only way forward really is to have Python actually implement boardgame.io's core engine. This is not as hard as before since a lot of the game spec is now very declarative, but you still have the problem that moves are written in JS, and the Python bot has to essentially re-implement that logic in order to play.

If we can get the moves into something that is a bit higher level (#455 might be relevant here), then we might actually be able to take a game spec and run it on a different language like Python or Rust.

[francoijs]

Another approach for AI enthusiasts could be to write the core game logic only in Python:

• use the Python native version for training a bot (as you said, some kind of python engine will be required here to execute correctly the moves and game events)
• have them translated to JS for boardgame.io (where performances matter less)

[yangboz]

https://github.com/francoijs/TicTacBot

any SGF file for tictactoe game?