search

CodeReclaimers / neat-python

Python implementation of the NEAT neuroevolution algorithm
BSD 3-Clause "New" or "Revised" License
1.4k stars 488 forks source link

fix: used initial observation the entire time #274

Open markste-in opened 11 months ago

markste-in commented 11 months ago

Fixes a bug where the initial observation is used over and over again.

Ensures the network is rendered even if the environment is not solved or aborted

markste-in commented 11 months ago

But I still feel like this example is broken. I can't get it to learn after almost 8000 gens.

I had a simpler example with the Lunar Lander and this package a few years ago that worked quite well. Tho I can't find that code rn. Maybe I find some time to troubleshoot it at some point.

@ntraft Can you get it to work?

ntraft commented 11 months ago

Oh, really?? As in, you were able to "solve" the Lunar Lander, or at least do well? I assumed that it never worked, when I started using this package ~9 months ago!

Did you see my issue #256 about the Lander example? Did you address the issue with the fitness function? I wonder what kind of fitness you were using when you had it working before?

I started using the Bipedal Walker instead, and it works kinda okay on flat terrain, but nowhere near as well as anything you see in these Evolution Strategies examples. But even to get to that point I had to fix the FeedForwardNetwork so that it will run nodes even if they don't have an input. I should probably also implement something to prevent the dangling nodes from appearing in the first place, like you did. I was trying to avoid that, at first, because I wasn't sure if it was intentional and wasn't sure about the best way to fix it. I put this project away for awhile but it's seeming like if I go much further with it I would have to address that issue.

It would be great to see it solve Lunar Lander! I just assumed it was too hard. I'm also curious, do you remember how important the speciation parameters were for your success? Tuning the speciation is seeming like a hot mess to me.

markste-in commented 11 months ago

No I wasn't able to solve it with the current code example even after thousands of generations. I think the fitness function is wrong or at least buggy but I haven't had enough time to debug it yet. I also think the fitness function is overly complicated. I will check my old hard dist because I am pretty sure that my own code from like a year a ago solved it kinda quickly. If i find the code I will add it to this Pull Request. The lunar lander is actually quite simple and should be solved quickly. Removing the dangling nodes doesn't make a big difference in this case since the network should never grow that big. In more complicated environments it helps to make the learning more "stable" in my experience

ntraft commented 11 months ago

Note some other issues that are suspicious and may be part of the problem:

I think elitism isn't working correctly, and best genomes might not be returned correctly, and this might also be negatively impacting speciation.

I'm going to try to investigate some of it this week.

allinduetime commented 1 month ago

These ever get solved?

Finebouche commented 1 month ago

@allinduetime check https://github.com/Finebouche/neat-gymnasium. We could think about merging the project here in the examples actually. But it look like @CodeReclaimers is not maintaining it ATM