search

AlexTheM8 / AiAi-AI

3 stars 0 forks source link

readme

AiAi AI

Super Monkey Ball AI using NEAT and YOLO SMB Logo

Contents

  1. Project Description
    1. Technologies Used
  2. Inspiration & Purpose
  3. Results
    1. Hardware Specs
    2. Graphs
  4. How To Use
    1. Dolphin Emulation Config
    2. Requirements
    3. Hotkey Config
    4. Controller Config
    5. Save State Setup
    6. Launch AI Agent
    7. Customizing Options
      1. Runtime Arguments
      2. Network
      3. Goal Detection
    8. Known Issues

Project Description

This repository contains the excessive work completed in an attempt to create an artificial intelligence agent in the game Super Monkey Ball. Check out this video on the project.

Technologies Used

This project utilizes two technologies: NEAT and YOLO. The game is expected to be played on the Dolphin Emulator.

NEAT

NEAT (Neural Evolution of Augmenting Topologies) is the primary mechanism utilized in this projects. To facilitate the evolution, the aiai_ai.py script runs the neat-python library. After a genome is initialized, a screenshot is taken at each interval in the script. This screenshot is then analyzed by the generated model and an action is determined accordingly. To determine the resulting state's fitness, the screenshot is evaluated to see if it is one of three states [TIME OVER, FALL OUT, GOAL].

all_states

In the GOAL state, the genome's fitness is determined by how quickly it reached the Goal, based on the following equation: 30 + 1.25 * time_remaining where the fitness of the GOAL state is between [30, 105].

The TIME OVER and FALL OUT states are more complicated to calculate. A problem arose of how to have a more granular fitness beyond a simple state check. To resolve this, it was determined to include a value representing how close the genome made it to the Goal before failing (see the YOLO section for more information). If a genome got as close to the Goal as possible, but still failed, it would be awarded a fitness score of 50. If the end-state was TIME OVER or FALL OUT this score would be modified by -25 points.

The highest fitness score during a given genome's runtime will be the score evaluated during population reproduction.

YOLO

YOLO (You Only Look Once) is used as a metric of determining how close a given genome is to the Goal. In short, the YOLO model detects the size and location of the Goal (if one is present), returning a bounding box of the Goal. The area of the bounding box is used to calculate the percentage of the screen the Goal occupies. If the Goal occupies at least 40% of the screen, the given genome will be given the max reward possible for TIME OVER or FALL OUT states.

Inspiration And Purpose

The design of this project was largely inspired by SethBling's MarI/O. The inspiration of starting this project came from my wife holding three World Records in Super Monkey Ball: Banana Mania speedruns (as of April 4th, 2022). I simply wanted to feel better about my inadequacy in this series.

The purpose of this project is to create a Super Monkey Ball AI that can reasonably beat standard stages in the game.

Results

AiAi AI was tested on three stages: Beginner Floor 1, Beginner Floor 2, and Beginner 7.

Hardware Specs

All results were conducted on the following specs

Windows 10
NVIDIA RTX 3060
AMD Ryzen 5 3600 6-Core Processor 3.95 GHz
16 GB RAM

With these specs, and default configurations, the AI is able to analyze the game at approximately 30 FPS, essentially every other frame of the game.

Graphs

So far, the AiAi AI has been tested on beginner floors 1, 2, and 7. These were chosen for their relative simplicity in layout, allowing for an easier evolution for the population.

Floor 1

floor_one_overview

Due to it's nature, Floor 1 is the simplest stage, only requiring the player to move straight ahead. Because of it's simplicity, it did not take long for the population to surpass the max expected fitness score of 101. Therefore, to gain further information about the population's evolution, a second run was conducted to see how high the average fitness for the population could achieve. The results are graphed below.

First Run

floor_one_graph

Second Run

floor_one_mean_graph

Floor 2

floor_two_overview

Beginner Floor 2 adds complexity to Floor 1 by having a gap in the middle of the stage. As the graph shows, it took the population a significant amount of generations to reach a higher maximum fitness. For this stage, the expected fitness threshold was 99.5. However, the population only reached a maximum of 98.76. After 323 generations, the process was halted.

floor_two_graph

Floor 7

floor_seven_overview

Floor 7 was a significant increase in complexity to test the overall capabilities of the system. The expected fitness threshold was set to 90. However, the population never officially crossed the goal. Despite this, there was significant progress made towards reaching the goal. The population almost reached the goal several times, but not close enough to get the fitness score out of the negatives.

floor_seven_graph

How To Use

To utilize this project, be sure to have a working version of the Dolphin Emulator installed on your machine, along with a ROM of Super Monkey Ball for GameCube. The project can be downloaded onto your local machine using the following command:

git clone https://github.com/AlexTheM8/AiAi-AI.git

Dolphin Emulation Config

For best results, the following settings should be configured for the Dolphin Emulator.

Requirements

The code of this project was programmed using Python 3.9.0 on a Windows 10 machine. It is currently unknown if this project is functional in any other Python version or OS.

Once downloaded, navigate into the project folder and execute the following command to install the Python dependency libraries:

pip install -r requirements.txt

If necessary, additional requirements can be installed by executing the following command:

pip install -r ./yolov5/requirements.txt
Additional instructions for CUDA-enabled devices If you are running a CUDA-enabled device, execute the following command: (For CUDA 10.2) ``` pip install torch==1.10.2+cu102 torchvision==0.11.3+cu102 torchaudio===0.10.2+cu102 -f https://download.pytorch.org/whl/cu102/torch_stable.html ``` (For CUDA 11.3) ``` pip install torch==1.10.2+cu113 torchvision==0.11.3+cu113 torchaudio===0.10.2+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html ```

Hotkey Config

Prior to executing the program, the virtual gamepad needs to be initialized. To do so, the controller.py must be configured executed.

Pulling up the Dolphin hotkey controls, navigate to the Save and Load State tab (see below).

dolphin_hotkey_default

Once there, execute the following command:

python controller -s load

This will set the controller to repeatedly press the virtual button designated to loading the game's state. Follow these steps to configure the Load State hotkey.

  1. In the Dolphin hotkeys menu, at the Device dropdown, select the virtual gamepad (multiple devices may be listed, so repeat steps 2-4 with a different device until the correct device is found)
  2. Click on the Load State Slot N (where N can be any number you choose. Remember this number) box
  3. Wait for Button 3 to appear in the box
  4. Repeat steps 2 & 3 a few times if it does not appear immediately
  5. You should then see Button 3 flash bold multiple times
  6. Terminate the controller.py process
  7. [OPTIONAL] Save the configuration as a Profile in Dolphin to avoid the need to repeat this process in the future

dolphin_hotkey_config

Controller Config

For the joystick controls, navigate to the Dolphin controller config menu.

dolphin_controls_default

To configure the joystick in Dolphin, follow these steps:

  1. Execute the command python controller -s up
  2. In the Dolphin controller config menu, select the Device corresponding to the virtual gamepad (as described in Hotkey Config, the device is not universally labeled, so steps 3-5 may need to be repeated until device is found)
  3. Under the Control Stick section, click the box next to Up
  4. Wait for Axis Y- to appear in the box
  5. Repeat steps 3 & 4 a few times if it does not appear immediately
  6. You should then see Axis Y- flash bold multiple times along with a visual representation of the joystick moving in the visual above
  7. Terminate the controller.py process
  8. Repeats steps 1-7 for each of the directions [down, left, right], replacing each instance of "up" with the corresponding direction (Axis Y+ for down, Axis X+ for left, Axis X- for right)
  9. [OPTIONAL] Save the configuration as a Profile in Dolphin to avoid the need to repeat this process in the future
  10. [OPTIONAL] To test the controller configuration, execute the command python controller -s random. The virtual joystick should be moving appropriately

dolphin_controls_config

Save State Setup

The evolution process requires a save state to be accessible as this is how the agent resets the game state upon starting a new genome. Please refer to Hotkey Config before configuring the save state.

  1. In the Super Monkey Ball game, navigate to the preferred stage (Note: if you are just starting out the game and do not want to play through the game to access later stages, please refer to this Wiki for more information on how to access all levels easily)
  2. Save the game state at the start of the stage (before any time passes) in the same Save Slot chosen in the Hotkey Config step (Note: for a more-precise save state, use the frame-advance TAS tools provided by Dolphin)
  3. Feel free to use the original code in controller.py to test if the save state (and corresponding hotkey) are configured appropriately (see Hotkey Config).

Launch AI Agent

Once all the previous steps are completed, launch the Super Monkey Ball ROM on Dolphin and execute the following command:

python aiai_ai

From there, you should see the program load up the save state and begin evolution.

IMPORTANT: After starting evolution DO NOT MOVE THE EMULATION WINDOW. If you want to move the window, restart the program.

Customizing Options

To customize behavior, there are some options and features available in the AiAi AI. These options include logging and stat-tracking as well as network customization options.

Runtime Arguments

To customize the amount of messages displayed when running, there are three logging options: [FULL, PARTIAL, NONE]. The FULL logging option (enabled by default) will display all available logs during evolution. The PARTIAL logging option will display the max fitness of the most-recently executed genome along with the default NEAT logs. The NONE logging option will only display the default NEAT logs.

python aiai_ai --logging full
                -l       partial
                         none

Along with the evolution logging, there is a stat-tracking document that saves the max fitness, mean fitness, and standard deviation of each generation (saved as a CSV file labeled stats.csv). This file is enabled by default, but can be disabled as an execution parameter.

python aiai_ai --stats
                -s

To speed up evolution, there is a feature where a given genome will be terminated early if it is not moving for too long. This is enabled by default, but can be disabled at execution. NOTE: feature should be disabled for stages that may require the player to wait.

python aiai_ai --zero_kill
                -z

On Windows systems, a magnification can be applied to the display. Because of this, there is a runtime argument to set the magnification of the used display.

python aiai_ai --window_scale 1.0
                -w            1.0

Scale must be set as a float (ex. 1.0, 1.25, 1.5, etc). NOTE: Scale has only been tested for 1.0 and 1.25

Network

As part of the NEAT library, the neural network can be customized in a number of ways. For a full description of all customization options, see the NEAT documentation. Some of these options in the config-feedforward document will be highlighted here.

In the [NEAT] section, the fitness_threshold can be adjusted for the given stage where the fitness can be between [-50, 105]. The pop_size, or "population size" can be customized, where this number will correspond to how many genomes are in each generation.

Under the network parameters category of the [DefaultGenome] section, the num_inputs will be dependent on your window configurations. Best way to know what the num_inputs should be set to is by running the AiAi AI program (see Launch AI Agent) and change the value to the received value, in case of execution failure. See the example below.

File "c:\neat-python\neat\nn\recurrent.py", line 27, in activate
    raise RuntimeError("Expected {0:n} inputs, got {1:n}".format(len(self.input_nodes), len(inputs)))
RuntimeError: Expected 1425 inputs, got 5967

In this example, num_inputs should be set to 5967

Goal Detection

The YOLO goal-detection is the primary method of determining a more granular fitness. The main method of customization is through the model training. Please refer to the YOLO documentation for more information on training customization. To ensure the goal-detection matches the YOLO model, the size parameter in the following line in aiai_ai.py should match the specifications made in training.

ref = model(cv2.cvtColor(img, cv2.COLOR_BGR2GRAY), size=320)

Known Issues

Below is a list of known issues with the current project.