search

BryanSWeber / CUNYAIModule

CUNYBot, an AI that plays complete games of Starcraft.
MIT License
6 stars 3 forks source link

BuildFAP problems. #111

Closed BryanSWeber closed 5 years ago

BryanSWeber commented 6 years ago

Using FAP simulations to determine what to build leads to an inappropriate creation of the wrong units.

This may be particularly visible in cases where the fights are very very lopsided, and therefore the simulation results are equally poor (or randomly so) in either direction.

Tested against Vadja- result was building lurkers and sunken colonies against a body of carriers and photon cannons - broadly useless. Perhaps the buildfap considered the anti-ground units to be good against the cannons and buildings since nothing was going to defeat the carriers anyway?

BryanSWeber commented 6 years ago

https://github.com/BryanSWeber/CUNYAIModule/commit/0faf0f5c61f08de835d4e2daca95bb27a9484f91

BryanSWeber commented 6 years ago

This patch below fixes a variety of the BuildFAP issues. Adds an element to the display in debug mode that announces if buildFAP is recommending AA or not.

https://github.com/BryanSWeber/CUNYAIModule/commit/aa9ee8d8e3d2f8bc85edaa84295f7d7e47f8f56c

BryanSWeber commented 6 years ago

Test results as of : https://github.com/BryanSWeber/CUNYAIModule/commit/e954ce06afec0fc96e9d2a151c8f7f0f9f29c98c

Row Labels Average of w (11/3) Prior Branch (10/14) Prior branch (10/23)
Carsten Nielsen 0.4 0 0.222222
Dave Churchill 0 0 0
Gaoyuan Chen 1 1 1
Hannes Bredberg 0 0.111111 0
ICELab 0 0 0.111111
Lukas Moravec 0.2 0.666667 0.666667
MadMixZ 0 0 0
Marine Hell 0.333333333 0 0.444444
NUS Bot 0.4 0 0.777778
OpprimoBot 1 0.777778 0.888889
Roman Danielis 0.166666667 0 0.222222
Steamhammer 0 0.111111 0.222222
Stone 1 0.666667 0.555556
Sungguk Cha 0.5 0.111111 0.222222
Tomas Vajda 0 0 0
WillyT 0 0 0
(blank)      
Grand Total 0.315217391 0.215278 0.333333
BryanSWeber commented 6 years ago

Note: the std deviation the number of wins is roughly:

N= number of games played = 916 = 144. P = average win % for bot, about 0.3 W = number of games won = PN (should have kept that info, dang.)

sqrt(144*P(1-P)) = std deviation of W, about +/-5.5 games, or 5.5/144 = +/- 0.038%

So the 10/23 and 11/23 branches are basically indistinguishable, but the 10/14 branch is def. worse.

BryanSWeber commented 5 years ago

BuildFAP has been entirely reworked and a ResearchFAP branch has been included. I have noticed these issues are noticibly improved.