Programmatic game control for integration with OpenAI Gym

[muety]

Hi all!

This is not particularly a feature request, but rather a discussion starter. If you consider it inappropriate, feel free to close this issue again.

Background

I have a project in mind to train a reinforcement learning-based AI agent to play Supertuxkart.

With reinforcement learning (RL), there usually is a programmtic agent, which (1) first observes its environment and then (2) acts by choosing the most promising one from a set of potential actions.

An environment observation could be a set of explicit parameters describing the current game state (e.g. player's world position, speed, selected powerup item, etc.) or, more generally, a game screenshot from which an implicit state is derived using machine learning / computer vision methods. An action could to press a certain button (e.g. "A" to steer to the left).

Creating a RL environment

A commonly used "framework" to implement reinforcement learning for games is the OpenAI Gym, for which you can write custom, game-specific so called environments, which wrap the game's inputs and outputs with a uniform API. There is a number of community-built environments for a variety of different 2D- and 3D games, including Obstacle Tower and the CARLA autonomous driving simulator.

One key requirement for each of these environment is the ability to programatically interact with the game environment through an API. To my understanding, the minimum required interactions between the Gym environment and the game are:

• Run a particular number of discrete time steps / frames (1..n) of the game and then pause again
• Reset the game environment (i.e., for instance, restart the current round and reset the player to the start)
• Be notified when the player has "lost" the game (i.e., for instance, dropped out of the track)

Given these API methods, the RL agent is able to run a loop of observing the environment, performing an action and observing the environment again one (or few) frames later to see the consequences - until no more progress is made or the game is considered "lost".

---

My questions to the STK community are:

• Is there someone who had already played a bit with STK and OpenAI Gym (or RL in general)?
• Is there already some kind of (Python) API to programatically control STK?
• If not, are there plans to come up with such?

I'm neither very familiar with game development and C++, nor with the STK code base in particular, sorry!

[Benau]

https://pypi.org/project/PySuperTuxKart/

[Benau]

assume you solved the problem