BackgroundThreadMethod() failed with exception

[smelfungus]

Hello, @colgreen! Thank you for that second sharpneat generation. Architecture, APIs, and ease of use are greatly enhanced! 🥇 I'm getting next frequent exceptions:

BackgroundThreadMethod() failed with exception [Non-negative number required. (Parameter 'index')]
System.ArgumentOutOfRangeException: Non-negative number required. (Parameter 'index')
   at System.Collections.Generic.List`1.RemoveRange(Int32 index, Int32 count)
   at SharpNeat.Neat.EvolutionAlgorithm.NeatEvolutionAlgorithm`1.TrimSpeciesBackToElite(Boolean& emptySpeciesFlag) in D:\sharpneat-refactor\src\SharpNeat\Neat\EvolutionAlgorithm\NeatEvolutionAlgorithm.cs:line 268
   at SharpNeat.Neat.EvolutionAlgorithm.NeatEvolutionAlgorithm`1.PerformOneGeneration() in D:\sharpneat-refactor\src\SharpNeat\Neat\EvolutionAlgorithm\NeatEvolutionAlgorithm.cs:line 226
   at SharpNeat.EvolutionAlgorithm.Runner.EvolutionAlgorithmRunner.BackgroundThreadMethodInner() in D:\sharpneat-refactor\src\SharpNeat\EvolutionAlgorithm\Runner\EvolutionAlgorithmRunner.cs:line 263
   at SharpNeat.EvolutionAlgorithm.Runner.EvolutionAlgorithmRunner.BackgroundThreadMethod() in D:\sharpneat-refactor\src\SharpNeat\EvolutionAlgorithm\Runner\EvolutionAlgorithmRunner.cs:line 242

Pointing to this NeatEvolutionAlgorithm TrimSpeciesBackToElite loop:

for(int i=0; i < speciesCount; i++)
{
    Species<T> species = _pop.SpeciesArray[i];
    int eliteSizeInt = species.Stats.EliteSizeInt;
    int removeCount = species.GenomeList.Count - eliteSizeInt;
    species.GenomeList.RemoveRange(eliteSizeInt, removeCount);

    if(eliteSizeInt == 0) {
        emptySpeciesFlag = true;
    }
}

Does it seem like species.Stats.EliteSizeInt may somehow become negative?

[colgreen]

Hi. Thanks for reporting. I will try to look into this over the weekend.

[colgreen]

From an initial quick look, I would guess that you have a genome evaluator that may be assigning negative fitness scores(?) Probably I need better checks on such things. Do you have code that recreates the problem that I can look at? Either a repository or a zip maybe?

[smelfungus]

@colgreen exactly, I'm building a bridge between sharpneat and OpenAI Gym environments, and a negative fitness score (reward) is a typical case there. With the first version of sharpneat I was adding the base value to raw OpenAI Gym fitness but I thought this second version allows negative fitness (that would be great!). I'll share my experiments soon, but nothing really tricky going there, FitnessInfo Evaluate(IBlackBox<double> box) is communicating to the OpenAI Gym environment, getting total episode reward back, wrapping it in FitnessInfo, and returning it.

[colgreen]

I'm building a bridge between sharpneat and OpenAI Gym environments, and a negative fitness score (reward) is a typical case there.

This new version of SharpNEAT does still require non-negative fitness scores, so you will need to apply some function that maps the OpenAI Gym fitness scores to NEAT scores. Adding a baseline fitness can work if there is a known minimum, or if not then the following is perhaps a nicer general purpose solution:

y = e^(x-1)

or,

y = x < 1 ? e^(x-1) : x

i.e.

non_negative_fitness = fitness < 1 ? Math.Exp(fitness-1) : fitness;

See plots here:

https://www.desmos.com/calculator/2w2maf2ueo

That will work as a general purpose solution. It may be beneficial to refine that approach for individual OpenAI Gym tasks, but I I think that would be fine as a starting point.

I am going to add some checks in sharpneat to detect negative fitness scores, and to give a more useful error message when that happens.

While I'm here I want to thank you for your sponsorship/support over the last year! It is much appreciated! :)

[smelfungus]

@colgreen y = x < 1 ? e^(x-1) : x is a great idea! It will allow both distinguishing between negative fitnesses and will provide almost linear positive part 👍🏻 I am glad to sponsor such a great project, that's the least I can do, thank you!

[colgreen]

Closing, as this has been addressed by adding checks to the fitness scores; see 2406e75f6d1a39eeb299cdd486953697a3d42bbf

Thanks.