Closed schrum2 closed 2 years ago
I think I found something similar to what needs to be in a text file from the documentation.
Very good! I also found an example on their GitHub of how to use this. See: https://github.com/CodeReclaimers/neat-python/blob/5590604240793da311d9fb44671510139d9574f8/tests/test_simple_run.py Specifically:
stats = neat.StatisticsReporter()
stats.save()
stats.save_genome_fitness(with_cross_validation=True)
This is a good first test, but we would probably want to add a filename as well.
According to this: https://github.com/CodeReclaimers/neat-python/blob/1d796adbb022cd3f84b4aecf678b26824e42b7f2/neat/statistics.py
I think the default file name is fitness_history.csv
, which we wouldn't always want
So I was able to get the fitness to save to csv file. The value for with_cross_validation
has to be false because it is not supported yet (I was getting an error and when I looked at their github, there was a comment about how it had to be false).
My next commit for this issue will focus more on the filename since I only focused on getting the thing to work and print to csv.
Attached is the fitness_history.csv produced from a type_target run where the desired number of blocks was 50 and the desired block was redstone block.
I realized the plotting has to happen for fitness based evolution so I placed the csv saving inside an if condition that only hits when INTERACTIVE_EVOLUTION is false.
We'll need a command line parameter for EXPERIMENT_NAME that is both the name of the csv file (or part of it) and the name of a results directory, kind of like MMNEAT
Before the default name for the produced csv file was "fitness_history.csv". Now it is dependent on some command line parameters, although I think one more can be added. Also now the csv file is saved in a sub directory related to its respective seed.
I will try to implement the visualize stuff as well.
Using visualize also work for visualize.plot_stats
but got error I do not fully understand for visualize.plot_species
. Also I had to run pip install graphviz
in anaconda before I got the visualize to work. Not sure if this means the requirements.txt has to be updated to reflect this or if there is a better way of using graphviz.
Update, visualize.plot_species also works, I just forgot how it works.
Update requirements.txt as well
Updated requirements.txt to have graphviz. I did not specify a specific version however.
We need to have the ability to make graphs with generations on the x-axis and max fitness score on the y-axis. Every generation we evaluate the whole population, and save the max fitness, which we plot for that generation. In a text file, this means you have the max fitness values in a column next to a column of the generation numbers. From that it is easy to plot in a variety of ways.
NEAT-Python already has several "reporters" that are attached to the population in
evolution.py
. They are responsible for periodically printing information to the console, but they can probably be used for this too ... I'm just not sure how.