This repository contains an RL environment based on open-source game Gameplay
Football.
It was created by the Google Brain team for research purposes.
Useful links:
For non-public matters that you'd like to discuss directly with the GRF team, please use google-research-football@google.com.
We'd like to thank Bastiaan Konings Schuiling, who authored and open-sourced the original version of this game.
Open our example Colab, that will allow you to start training your model in less than 2 minutes.
This method doesn't support game rendering on screen - if you want to see the game running, please use the method below.
This is the recommended way to avoid incompatible package versions. Instructions are available here.
sudo apt-get install git cmake build-essential libgl1-mesa-dev libsdl2-dev \
libsdl2-image-dev libsdl2-ttf-dev libsdl2-gfx-dev libboost-all-dev \
libdirectfb-dev libst-dev mesa-utils xvfb x11vnc libsdl-sge-dev python3-pip
python3 -m pip install --upgrade pip setuptools psutil
First install brew. It should automatically install Command Line Tools. Next install required packages:
brew install git python3 cmake sdl2 sdl2_image sdl2_ttf sdl2_gfx boost boost-python3
To set up pygame
, it is also required to install older versions of SDL:
brew install sdl sdl_image sdl_mixer sdl_ttf portmidi
python3 -m pip install gfootball
git clone https://github.com/google-research/football.git
cd football
Optionally you can use virtual environment:
python3 -m venv football-env
source football-env/bin/activate
The last step is to build the environment:
python3 -m pip install .
This command can run for a couple of minutes, as it compiles the C++ environment in the background.
python3 -m gfootball.play_game --action_set=full
Make sure to check out the keyboard mappings. To quit the game press Ctrl+C in the terminal.
In order to run TF training, you need to install additional dependencies
python3 -m pip install --upgrade pip setuptools
python3 -m pip install tensorflow==1.15.*
or
python3 -m pip install tensorflow-gpu==1.15.*
, depending on whether you want CPU or
GPU version;python3 -m pip install dm-sonnet==1.*
;python3 -m pip install git+https://github.com/openai/baselines.git@master
.Then:
academy_empty_goal
scenario, run
python3 -m gfootball.examples.run_ppo2 --level=academy_empty_goal_close
academy_pass_and_shoot_with_keeper
scenario, run
python3 -m gfootball.examples.run_ppo2 --level=academy_pass_and_shoot_with_keeper
In order to train with nice replays being saved, run
python3 -m gfootball.examples.run_ppo2 --dump_full_episodes=True --render=True
In order to reproduce PPO results from the paper, please refer to:
Please note that playing the game is implemented through an environment, so human-controlled players use the same interface as the agents. One important implication is that there is a single action per 100 ms reported to the environment, which might cause a lag effect when playing.
The game defines following keyboard mapping (for the keyboard
player type):
ARROW UP
- run to the top.ARROW DOWN
- run to the bottom.ARROW LEFT
- run to the left.ARROW RIGHT
- run to the right.S
- short pass in the attack mode, pressure in the defense mode.A
- high pass in the attack mode, sliding in the defense mode.D
- shot in the the attack mode, team pressure in the defense mode.W
- long pass in the the attack mode, goalkeeper pressure in the defense mode.Q
- switch the active player in the defense mode.C
- dribble in the attack mode.E
- sprint.Run python3 -m gfootball.play_game --action_set=full
. By default, it starts
the base scenario and the left player is controlled by the keyboard. Different
types of players are supported (gamepad, external bots, agents...). For possible
options run python3 -m gfootball.play_game -helpfull
.
In particular, one can play against agent trained with run_ppo2
script with
the following command (notice no action_set flag, as PPO agent uses default
action set):
python3 -m gfootball.play_game --players "keyboard:left_players=1;ppo2_cnn:right_players=1,checkpoint=$YOUR_PATH"
We provide trained PPO checkpoints for the following scenarios:
In order to see the checkpoints playing, run
python3 -m gfootball.play_game --players "ppo2_cnn:left_players=1,policy=gfootball_impala_cnn,checkpoint=$CHECKPOINT" --level=$LEVEL
,
where $CHECKPOINT
is the path to downloaded checkpoint.
In order to train against a checkpoint, you can pass 'extra_players' argument to create_environment function. For example extra_players='ppo2_cnn:right_players=1,policy=gfootball_impala_cnn,checkpoint=$CHECKPOINT'.