Testing Results: Learning

[BryanSWeber]

Describe the bug Learning is super slow with the GA, even after improving it around 1k games.

I suspect it is either broken, or there is simply too much noise. The win rate does not even trend upward ( The enemies also learn, possibly faster.) Some bots in the testing cycle always lose, adding noise. There are several maps, and many bots to play against, and multiple openings along each of which I stratify the learning results.

Let's redo this experiment with 3 bots, a T, a P, and a Z. Only 1 map, fighting spirit.

MatejIstenik LukasMoravec PurpleSwarm

--

[BryanSWeber]

In the aforementioned sub-experiment: after a few hundred games, the parameters are roughly as follows:

And the fitness remains relatively consistent as well (an ad-hoc but convex combination of both score and victory):

I have been thinking this has been a result of me training the unit weights alongside the bots decisions. The unit weights is a vector of several hundred (mostly noise) parameters. It is probably the case that this simply dominates the opening parameters and the GA cannot learn fast enough to train both of these alongside one another.

I will give it a few more days to train (since it is costless to me) and prioritize simply fixing the easily managable: https://github.com/BryanSWeber/CUNYAIModule/issues/211

If it has not converged to a noticeably better level of fitness by then, I will disable learning of unit weights or train it independently on a more concerted loop.

[BryanSWeber]

Training has been focused on a specific loop, no unit weights are being learned. We're using a simple GA for learning. Setup: Fighting spirit. MatejIstenik LukasMoravec PurpleSwarm

After about 100 games (two samples of 100 games each, trained in an embarrassingly parallel format) there's really only weak evidence of convergence on any values or improvement in win rate in either set. Will check again after about 500 games, but should investigate this code carefully. The trend should at least be reliably positive within the first 100 games and I would expect a change in the opening types over time!

Sample 1: Sample 2:

[BryanSWeber]

After about 200 games, I see no strong evidence of convergence. There ought to be a correlation between the number of games and the win rate, win rates CAUSING more attempts. It is odd that there is not such a correlation.

I'll have to reflect on this and check for bugs.

[BryanSWeber]

Ha! found a major bug: c41be7063c2164b023d6690022869574d6854ad2

I did not use "safename" on the enemy bot name or map, so the only match was race :(

[BryanSWeber]

Tested, and this instantly resolved the problem, bot snapped into doing a 7-pool against Lucas M without deviation and obtained a 100% winrate for 4 consecutive games.

A good hundred CPU hours spent getting a good sample size ready, I guess!

[BryanSWeber]

Just wanted to show that it has worked, and the tit for tat style remains preferential!