search

schrum2 / MM-NEAT

Modular Multiobjective (Hyper) Neuro-Evolution of Augmenting Topologies + MAP-Elites: Java code for evolving intelligent agents in Ms. Pac-Man, Tetris, and more, as well as code for Procedural Content Generation in Mario, Zelda, Minecraft, and more!
http://people.southwestern.edu/~schrum2/re/mm-neat.php
Other
50 stars 20 forks source link

Unit Tests for ChangeBlocksFitness #907

Closed schrum2 closed 1 year ago

schrum2 commented 1 year ago

We need to understand how this is working better and make unit tests for ChangeBlocksFitness that both do and don't use the minecraftChangeBlocksMomentum option.

In these tests, it would be good to test out a flying machine and figure out what sort of fitness values result, and see if they seem appropriate. However, for the tests with no flying, we should be able to rigorously determine the appropriate values in advance.

TjRaffert commented 1 year ago

I am going to make a 3x3x3 version and run it. I think this will help us make more sense of how its evaluating.

TjRaffert commented 1 year ago

I spent some time this morning examining the results of MomentumChangeBlocks in 5x5x5 and 3x3x3. The 5x5x5 had a slight increase in fitness. from 190 to a little over 200. I think this just maxed out the fitness with oscillating machines. Since it is saving it as xml files, and it is hard to narrow down the specific machine with the technique you showed me earlier. The 3x3x3 had a larger increase from 80 to 140 in fitness. I believe it made flying machines. The test is still running so I tried searching the test for flying machines. I couldn't find one actually flying but I did find some left over blocks that were probably flying machines that got cut in half.

schrum2 commented 1 year ago

This is a reason to prioritize issue #913 . Once that issue is done, you can run some new experiments. Perhaps for a first attempt at #913, you can do something simple, like say we should save the shape whenever the fitness is above some certain threshold, like 120. This would at least let us look at the high scoring shapes easier, though we would probably change the save criteria later.

TjRaffert commented 1 year ago

I created a postSpawnEvaluateBlocksMomentumChangeBlocks.bat to evaluate our results. We are getting several false positives. We are going to try and reward the flying machines more by squaring the distance.

TjRaffert commented 1 year ago

The tests are created but they will probably need some adjusting after we get the calculation fully figured out.

schrum2 commented 1 year ago

Unit tests here are now crashing because they are trying to save a file with the blocks (because of #913 ). To fix this, it is very important that every Parameter initialization has "netio:false". I've made a start ... complete the rest (after merging my branch)

TjRaffert commented 1 year ago

I was able to get all the unit tests to pass on a maven build so it should be good to go.