search

JernejHabjan / TrumpDefense2020

Visual representation of RTS game, supported by deep reinforcement learning algorithm Alpha Zero written in python.
https://jernejhabjan.github.io/TrumpDefense2020/
MIT License
6 stars 1 forks source link
alpha-zero artificial-intelligence deep-learning game game-development monte-carlo-tree-search neural-network real-time-strategy reinforcement-learning rts-game unreal-engine-4

readme

license

Trump Defense 2020

Author: Jernej Habjan

Title:

Versions

Navigation

Quick overview

Visual representation of RTS game, supported by deep reinforcement learning algorithm Alpha Zero written in python.

Features:

Startup Guide

Visual Studio

install Visual Studio

Unreal Engine

PyTorch

Install Python dev environment https://developer.nvidia.com/how-to-cuda-python

type this in anaconda prompt with admin rights

conda update conda
conda install numba
conda install cudatoolkit
conda install -c peterjc123 pytorch cuda90

TensorFlow and CUDA

Install cuda:

vertify cuda installation:

Graphviz and pydot:

conda install graphviz
conda install pydot

Download Graphviz executable from Here

Add path to environmental variables and !important! restart Pycharm

C:\Program Files (x86)\Graphviz2.38\bin

Conda export

to export form conda cmd

conda list --explicit > requirements.txt

to install on other machine

conda create --name myenv --file requirements.txt
activate myenv

Requirements are in _Files folder as requirements.txt

Python Alpha Zero algorithm

Defining actions

actions = ["idle", "up", "down", "right", "left", "mine_resources", "return_resources", "attack", "find_enemy", "npc", "rifle_infantry", "town_hall", "barracks", "sentry", "mining_shack", "continue_building"]

That sums up to 16 actions

Defining game rules:

Defining win Conditions

1. Create 3 builders:

Point is for algorithm to learn to select town hall and execute action 'npc' before opponent does Actions:

actions = ["idle", "up", "down", "right", "left", "npc"]

Action sequence:

npc
npc
npc

2. Pick up minerals:

Neural network must select town hall, execute action 'npc' to create workers and individually move them towards minerals and pick minerals up Actions:

actions = ["idle", "up", "down", "right", "left", "mine_resources"]

Action sequence:

npc
(while not at minerals)
    move (in apropriate direction)
gather_resources

3. Return minerals:

Player would have to successfully return picked up minerals to resource drain like Town Hall or Mineral Mine. Actions:

actions = ["idle", "up", "down", "right", "left", "mine_resources", "return_resources"]

Action sequence:

npc
(while not at minerals)
    move (in apropriate direction)
gather_resources
(while not at resource drain)
    move (in apropriate direction)
return_resources

4. Build barracks (0 initial money):

Player receives 1 TownHall and 1 NPC at start with 0 money and has to gather enough resources to build barracks. Actions:

actions = ["idle", "up", "down", "right", "left", "mine_resources", "return_resources", "barracks", "continue_building"]

Action sequence:

(while not enough minerals for barracks)
    (while not at minerals)
        move (in apropriate direction)
    gather_resources
    (while not at resource drain)
        move (in apropriate direction)
    return_resources
barracks
(while barracks not built)
    continue_building

5. Gather 1000 minerals:

Point is to set timeout to lower number, which would force player to build economy (building new workers) while gathering minerals. It would need to place mining_shack closer to minerals to more efficiently gather minerals. Actions:

actions = ["idle", "up", "down", "right", "left", "mine_resources", "return_resources", "npc", "mining_shack"]

Action sequence:

(while enough minerals)
    npc OR mining_shack
(while not at minerals)
    move (in apropriate direction)
gather_resources
(while not at resource drain)
    move (in apropriate direction)
return_resources

4. Build Rifle unit from Barracks (0 initial money):

Player receives 1 TownHall and 1 NPC at start with 0 money and has to gather enough resources to build Barracks and from it Rifle unit. Actions:

actions = ["idle", "up", "down", "right", "left", "mine_resources", "return_resources", "rifle_infantry", "barracks", "continue_building"]

Action sequence:

(while not enough minerals for barracks)
    (while not at minerals)
        move (in apropriate direction)
    gather_resources
    (while not at resource drain)
        move (in apropriate direction)
    return_resources
barracks
(while barracks not built)
    continue_building
(repeat step for gathering minerals until enough for Rifle unit)
rifle_unit

4. Destroy enemy (0 initial money):

Player receives 1 TownHall and 1 NPC at start with 0 money and has to gather enough resources to build Barracks and from it Rifle unit. Then has to move rifle unit towards enemy and with it keep attacking enemy units until they are destroyed. Actions:

actions = ["idle", "up", "down", "right", "left", "mine_resources", "return_resources", "attack", "find_enemy", "npc", "rifle_infantry", "barracks", "continue_building"]

Action sequence:

(while not enough minerals for barracks)
    (while not at minerals)
        move (in apropriate direction)
    gather_resources
    (while not at resource drain)
        move (in apropriate direction)
    return_resources
barracks
(while barracks not built)
    continue_building
(repeat step for gathering minerals until enough for Rifle unit)
rifle_unit
(while not at enemy)
    move (in apropriate direction)
(while enemy is alive)
    find_enemy
    (while found enemy is alive)
        attack

Publishing

Steam Api

Fix port forwarding on your router

Windows firewall

It's located in

UE4EngineSourceCode/engine/binaries/win64/UnrealFrontend.exe

  • if you don't have it - visual studio - class view - unreal frontend - right click and build, and then you can access it
  • open UnrealFrontend.exe
  • add custom Launch profile:
  • Project - browse to your project and select Uproject File
  • Do you wish to build -try without , but if errors - yes - in development

Run tensorboard

cd C:\Users\Jernej\venv\Scripts tensorboard --logdir=C:\TrumpDefense2020\TD2020\Content\Scripts\td2020\models\logsTD2020NNet

Sources:

Plugins used

Video tutorials followed

License

MIT License

Copyright (c) 2018 Jernej Habjan

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.