Fitness based evolution of a snake that fills up as much of the confined space as possible

[schrum2]

Time to learn about fitness functions.

In fitness_functions.py define a fitness function following the mold of the others that exist. Call it occupied_count. It counts the number of blocks within the 3D space (for a particular corner and given x/y/z range values) that are not AIR blocks, in other words not empty.

Next, add whatever code is necessary to make fitness_cppn_evolution.py capable of evolving snakes. Basically, just look to interactive_cppn_evolution.py as an example, and be sure to define a function self.query_cppn as was very recently done in that file (make sure you get the latest from @MuellMark 's branch).

[schrum2]

I believe this works now, but I'm not having success reaching the max fitness. The command I'm testing with is this:

python .\main.py --INTERACTIVE_EVOLUTION=False --FITNESS_FUNCTION=occupied_count --EVOLVE_SNAKE=True --KEEP_WORLD_ON_EXIT=True --REDIRECT_CONFINED_SNAKES=True --BLOCK_LIST_EVOLVES=False

Systematically test with shorter max snake lengths to see if we can eventually succeed, and report the results here. I think we should also see if we can succeed by keeping the current max length, but making the population size bigger than 10 (also curious to see how much this slows things down)

[richeyme]

I tried with the max snake length of 50 and this is what I got:

This it found a champion but it was cleared as soon as it finished

[richeyme]

Now the structures won't go away after finding the champion. I had to scale down the max snake length to 40 because 50 kept taking too long and crashing. This is the champion for 40

[schrum2]

Here is a fitness graph: It's hard to sustain improvement. I'm going to try using a larger population to see if that helps.

[schrum2]

I tried upping the population size to 30 and can't seem to get better I'm hoping the new issue #56 will help. Otherwise, there might be a real problem her we need to address.

[richeyme]

Ran the fitness evolution for the snakes for a total of 1000 generations with a population size of 45 and the max snake length of 100 and there were still no champions. I saw the fitness get up to 62 but yet there were no signs of stopping

[schrum2]

I tried using elitism. When I run with this command

python main.py --FITNESS_FUNCTION=occupied_count --KEEP_WORLD_ON_EXIT=True --SAVE_POPULATION=True --BASE_DIR=snake --EXPERIMENT_PREFIX=occupied --INTERACTIVE_EVOLUTION=False --RANDOM_SEED=0 --POTENTIAL_BLOCK_SET=machine --DESIRED_BLOCK=REDSTONE_BLOCK --USE_ELITISM=True --EVOLVE_SNAKE=True --CONFINE_SNAKES=True --REDIRECT_CONFINED_SNAKES=True

I get these results:

This is different, but still not successful. Basically, elitism seems to force progress for a while, but then the fitness plummets. I'm not 100% sure why, but I think that NEAT Python has settings related to extinction and resetting. Specifically, I think that a species "goes extinct" if it gets stuck for a certain number of generations with no progress. I also think that if all species go extinct, that the evolution completely resets from scratch rather than failing, but there is an easy setting to change this.

This at least explains why we can evolve so long without seeming to make any progress ... we keep resetting. Next question is what to do about this. This problem should not be this challenging, and if we can't overcome this we will have trouble overcoming greater problems.

[schrum2]

By the way, the snake length evolution was trying to achieve in the previous example is 100, the default snake length. Looks like the best we did was a bit above 50

[richeyme]

So far we see that elitism is working but there is a weird thing happening with average fitness: some are -inf which doesn't seem to be making sense. A change was made so that it would add a corner into the world instead of making the fitness - inf and this is being tested currently.

[schrum2]

It looks like using elitism with a large enough population finally worked. Since the max snake length is 100, that was our fitness target. However, we were also using the REDIRECT setting which makes the problem a bit easier. Still, a victory is a victory! Here is what the winner looks like: Notice that it kind of cheated by using pistons. A piston can actually take up 2 spaces if it is extended, but that arguably just shows how clever evolution is. Here is the fitness plot over time: We still have lots of extinctions happening, but the fitness gradually grows and eventually passes the threshold. We could probably do better, but I'm happy enough with this result to close the issue.

[schrum2]

This image was also produced: I think it indicates that the whole population only consisted of one species at a time, and that species changed at each extinction event.

[schrum2]

One more note. The command that led to the successful result was this one:

python main.py --FITNESS_FUNCTION=occupied_count --KEEP_WORLD_ON_EXIT=True --SAVE_FITNESS_LOG=True --BASE_DIR=snake --EXPERIMENT_PREFIX=occupied --INTERACTIVE_EVOLUTION=False --RANDOM_SEED=0 --POTENTIAL_BLOCK_SET=machine --USE_ELITISM=True --EVOLVE_SNAKE=True --CONFINE_SNAKES=True --REDIRECT_CONFINED_SNAKES=True --POPULATION_SIZE=100

However, I wonder if we should try without evolving the block list, since that might actually be affecting the species. I guess I'll re-open this. Use this same command, but set BLOCK_LIST_EVOLVES to false.

[richeyme]

I was able to get a champion yesterday a bit past 5 PM, which was a lot faster than I thought it was going to take.

I got these graphs:

I would say that these looks very similar (almost identical) to your findings.