Maplestory AI Trainer

Uses screen captures, OCR, and Reinforcement Learning to optimize training on a specific map in Maplestory

Demo

Following demo is completely run via AI.

About The Project

The projects functions by first taking a screenshot of the maplestory screen. It then crops out the exp and health locations of the screen. The reward is comprised of amount of exp gained - amount of health lost. There are 3 multi discrete actions the AI can take and any one time using.

1. Moving: left, right, up down
2. Attacking: basic attack, power attack (lucky seven)
3. Misc: Pick up item, Jump

At first, the model was done training after about 300 steps. It was just walking into a wall and attaching constantly. Researched a bit and lowered the learning rate, adjusted the gamma, and raised the batch size. This produced decent results.

Creates a customer environment using stable baselines3. Utilzes the PPO modules with CnnPolicy as it's good for pixel-based input.

Getting Started

Prerequisites

• Install python 3.11
• Install Tesseract-ocr source

Installation

1. Clone the repo

   git clone https://github.com/GrahamMThomas/MapleAITrainer.git

2. Create virtualEnv and install requirements

   python -m venv venv
   ./venv/scripts/Activate
   pip install -r requirements.txt

3. Run Training

   python train.py

• check_env.py - Checks env for method signatures and input outputs are valid
• test_env.py - Runs random commands on your environment to ensure it works
• run_latest.py - Loads the latest_model and runs the environment using it to drive commands w/o training

Running Metrics

Launch Tensorboard

tensorboard --logdir .\logs\

Had to backtrack around 15k steps as training was invalid

(back to top)

Roadmap

• [ ] Use ratios to load health, exp, and maple admin locations. Currently breaks if you go to another computer which runs at a different resolution
• [ ] Preprocess data. Currently we send a lower rez screenshot to the model as input. I think there is a lot of noise and stuff like ladders/steps aren't detected well. By using OCR to locate player, enemies, ladders, steps, floors etc, I may be able to improve model performance.
• [ ] Use minimap to punish the AI for staying in one area too long as enemies may have accumulated in other areas.

See the open issues for a full list of proposed features (and known issues).

(back to top)

(back to top)