scidash / neuronunit

A package for data-driven validation of neuron and ion channel models using SciUnit
http://neuronunit.scidash.org
38 stars 24 forks source link

Optimization results #183

Open russelljjarvis opened 6 years ago

russelljjarvis commented 6 years ago

If you scroll down you can see a visualization of the optimization results in this notebook: https://github.com/russelljjarvis/neuronunit/blob/dev/neuronunit/unit_test/test_ga_versus_grid.ipynb

Looking at the mean and std deviation of the gene population versus time, is a bit troubling. Showing that after 3 generations mean errors get worse not better, with more GA evolution. However the reason could be that the gene population keeps exploring, to avoid stagnation. It's quite possible that this is normal behavior of GA, as long as the hall of fame contains results that are very similar to grid search, then we can conclude that 3 or 4 generations is enough for the HOF to collect the best gene, and therefore by limiting GA evolution to a maximum of 4 generations run times should be significantly shorter.

I viewed similar graphs previously when the algorithm was nsga based selection I wish now, that I have simply compared the HOF result to the grid search result, as a way of doubly validating the GA result. I will do this tomorrow.

There are three unique error surfaces, for three out of 5 unique neurolex/neuroelectro cell datums. New lines 90-92 write nml files for each optimized cell.

https://github.com/russelljjarvis/neuronunit/blob/dev/neuronunit/unit_test/pipe_entry_point.py#L90-#L92

-rw-r--r-- 1 russell users 1730 May 17 17:54 nlex_id_100201.nml
-rw-r--r-- 1 russell users 1729 May 17 18:12 cell_nlex_idnifext_50.nml
-rw-r--r-- 1 russell users 1730 May 17 18:14 nlex_id_nifext_50.nml
-rw-r--r-- 1 russell users 1729 May 17 18:32 cell_nlex_idnifext_120.nml
-rw-r--r-- 1 russell users 1730 May 17 18:34 nlex_id_nifext_120.nml
-rw-r--r-- 1 russell users 1729 May 17 18:54 cell_nlex_id830368389.nml
-rw-r--r-- 1 russell users 1730 May 17 18:56 nlex_id_830368389.nml
-rw-r--r-- 1 russell users 1729 May 17 19:13 cell_nlex_idsao471801888.nml
-rw-r--r-- 1 russell users 1730 May 17 19:14 nlex_id_sao471801888.nml