Error in ga.gd and strange behaviour in the ai

[JDSECO]

I have a problem with the code and a confusion. The problem is that when the car arrives at the threshold, and the window for selecting the new threshold appears, when I write it and confirm the change, it gives the following error in ga.gd: push_error("mass extinction"); breakpoint In the version that is compiled and can be downloaded from here, this error does not occur.

And the confusion comes with the fact that when the generations are advanced (56-70) the cars lose everything they have learnt and return to the behaviour of the beginning without having completed the circuit.

The version of Godot I use is 3.2.2 stable.

[pastra98]

Hi, thanks for pointing this out! I just fixed it.

Why it happened

Turns out the issue was with the car_main.gd (this is the old version) script. During the physics process, after calling ga.next_timestep(), a check happens whether enough time has passed to start a new generation. If so, ga.evaluate_generation() is called first, which in turn first calls finish_current_agents(). finish_current_agents() takes the fitness from the agents (agent.gd, which implements the neural net specified by the genome and uses it to control car.gd) and assigns it back to the genome. Then it removes the bodies from the scene. After this happens, ga.evaluate_generation() calls update_curr_species(). This method finally updates the species, by calling species.update(). The species update basically assesses the fitness of its members, and if the species did not get to spawn any members during the ga.next_generation() call that happened before, it gets killed off. If all species get killed during update_curr_species(), the mass extinction error is raised.

Now, the bug happens because I made a mistake in car_main.gd that causes ga.evaluate_generation() to be called twice, skipping an entire generation. By being called twice in rapid succession, ga.evaluate_next_generation() causes the species to check their alive members. Because ga.next_generation() never got called before that happens, none of the species have members, and they all get killed off. In short, it is because generations are evaluated twice, which causes a bunch of bugs.

The mistake I made was to put ga.next_generation() into a conditional after opening the splash screen. If you click “continue with new threshold”, the update loop is unpaused, and evaluate_generation() is called again without having updated the species. I simply removed that conditional, and it works now.

Why species get worse after many generations

Regarding the second part of your question, I assume this is because species are killed off when they stop improving for a few generations. Take a look again at the species.update() method: Every generation this method gets called for all currently alive species. There is a check whether the current species leader (which is the most fit member of the species, unless the setting is overwritten in params) has achieved a higher fitness than the all time best of that species. If not, num_gens_no_improvementgets incremented.

num_gens_no_improvementis again checked during the update in the next generation, and if it exceeds the parameter, the species is killed off. My guess is that most species kind of stop improving after having “mastered” the track, and when the leader can't beat the fitness of previous generations, even the best species gets killed off.

You could try to increase allowed_gens_no_improvement in the params file, but that again would prevent killing off stalling species early in the process, so things might take longer to evolve.

Again, thank you for the bug report 🙂. I got very excited when I saw that email today, and I’m glad that people stumble upon this project. I’m super busy with university stuff at the moment, but I plan to continue working on this in the summer – there are still 2 branches that I want to merge once I completed the features. Also I should really implement multithreading for the ga.next_timestep() method. Let me know if you encounter any other problems in the meantime.

[pastra98]

Forgot to close the issue, but you can still ask questions here

[JDSECO]

Many thanks to you. If it wasn't for your code, I would never have been excited to learn more about neural networks. In fact, I've been analyzing your code for several weeks now, adapting it to a project until I ran into this problem. Since I am just starting to understand how these neural networks work, I found it impossible and frustrating not to be able to solve the problem myself. That's why I decided to write to you. But I want you to know that this is a great project, without it I would not have been involved in learning. And I value immensely your great effort, although it may seem to you that your work is not valued as it deserves. Keep up the good work. Best regards.

[pastra98]

Omg, thank you so much for the kind words. I’m really glad that I’ve put this out on github, It’s been a very positive experience so far.

I’m flattered by your praise, just wanted to point out that I’m also still a beginner, and that I’ve made weird choices in the codebase where there’s a lot of back and forth between objects, and state mutations that are not obvious. I’m sure you already recognized that, I just wanted to put that out as a disclaimer.

Nonetheless, your motivation to dig through my stuff inspires me - maybe the fact that it is not a massive library written by professionals such as gym.openai makes it more accessible. If you want to gain a deeper understanding of how I implemented NEAT, I would recommend you to check out the NEAT chapter in “ai techniques for game programming” by mat buckland.

I think that coding genetic algorithms in a high-level language such as gdscript is not scary at all. Professional libraries use tons of optimizations in neural networks, but really you can get decent results by just doing it the obvious way: having simple neurons that aggregate sums, put them into an activation function, and passing that result further on. Just don’t expect crazy results. The project that I originally wrote this library for didn’t work out with NEAT after all (have the ai control creatures via giving off impulses). But it has been a great first step into genetic algorithms and writing larger programs in general. And it is super rewarding to see the agents improve after a while.

Edit: Forgot to mention it, but a book that really inspired me is "The nature of code" by Daniel Shiffman. It shows how you can achieve really cool life-like behavior without complex AI stuff. He also has a great youtube channel.