Ultimate Volleyball is a multi-agent reinforcement learning environment built on Unity ML-Agents.
See 'Ultimate Volleyball Micro-Machine Learning Course' for an updated step-by-step micro-course.
Version: Up-to-date with ML-Agents Release 19
ultimate-volleyball
Unity project.ultimate-volleyball
project in Unity (Unity Hub β Projects β Add β Select root folder for this repo).VolleyballMain
scene (Project panel β Assets β Scenes β VolleyballMain.unity
).Heuristic Only
, ensure that the Behavior Type is set back to Default
(see Heuristic Mode).ml-agents
.mlagents-learn <path to config file> --run-id=<some_id> --time-scale=1
<path to config file>
with the actual path to the file in Step 3tensorboard --logdir results
to observe the training process. For more detailed instructions, check the ML-Agents getting started guide.
To enable self-play:
config/Volleyball_SelfPlay.yaml
(ML-Agents Documentation)ResolveEvent()
in VolleyballEnvController.cs
.Goal: Get the ball to bounce in the opponent's side of the court while preventing the ball bouncing into your own court.
Action space:
4 discrete action branches:
Observation space:
Total size: 11
Reward function:
The project contains some examples of how the reward function can be defined. The base example gives a +1 reward each time the agent hits the ball over the net.
The following baselines are included:
Volleyball_Random.onnx
- Random agentVolleyball_SelfPlay.onnx
- Trained using PPO with Self-Play in 60M stepsVolleyball.onnx
- Trained using PPO in 60M steps (without Self-Play)