search

schrum2 / MM-NEATv2

MM-NEAT version 2.0 is no longer supported. Please get MM-NEAT 3+ from https://github.com/schrum2/MM-NEAT
Other
11 stars 5 forks source link

Implement HybrID extensions #297

Open schrum2 opened 7 years ago

schrum2 commented 7 years ago

After implementing HybrID ( #258 ) we should implement all of the extensions to HybrID described in this paper: http://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0174635&type=printable

schrum2 commented 7 years ago

I've been looking at Auto-Switch-HybrID, and one problem with applying it is that it is intended for domains with reliable fitness evaluation. Nearly every domain that MM-NEAT is applied to has noisy fitness evaluation, meaning that the same agent may earn different scores on subsequent evaluations. As a result, the fitness plots with MM-NEAT tend to fluctuate a lot, making a strict application of the auto-switch criteria inappropriate.

However, rather than basing the switch point on the champion fitness, we can base it on a recency-weighted average of the fitness. This gives a more reliable measure of the fitness with less fluctuation.

There is a place in my code where a recency-weighted average (RWA) of each fitness function is already calculated: TUGNSGA2 ... look at the moveRecencyWeightedAveragesTowardsAverages method. However, this is a completely separate evolutionary algorithm that extends the regular NSGA2 that is used by default everywhere else.

I'm not sure what the best way to implement auto-switch-hybrID is. We could add tracking of RWAs to NSGA2 by default, but this seems excessive since it will not be used in most cases. An alternative option is to create a new EA that extends NSGA2 the way TUGNSGA2 does, but this seems a bit burdensome.

We'll talk about possible approaches later, but these are at least some options.

In the meantime, just focus on testing the offset-hybrID genotype encoding.

schrum2 commented 7 years ago

Running some short experiments (batch files) that use offset hybrID is appropriate. Modify the existing Tetris hybrID batch files, but set the switch point to a small number so you don't have to wait long for it to happen.

schrum2 commented 7 years ago

Also, about the comment above, once the batch files seem to work (do one for both "Raw" and "Features") change the preset hybrID switch point to 100, and commit the batch files.

twerisa commented 7 years ago

OffsetHybrID batch files seem to work properly - switched from HyperNEAT to OffsetHybrIDGenotype at the correct point and is running seamlessly.

schrum2 commented 7 years ago

I'm removing the "complete before end of SCOPE" label since Offset HybrID seems to work, but I'm going to leave this issue open because Auto-Switch-HybrID is not implemented yet. If you finish everything else, then you can take a look at it, but I suspect I'll be leaving this issue open for myself or some other student to solve in the future.