SES: brute force branching parameter settings

richelbilderbeek commented 7 years ago

[x] create a way to create random parameters objects
[ ] profile why the program is slow for 10K individuals
[x] define operator<< for parameters
[x] in main, show the parameters, using operator<<
[x] detect branching automatically
[ ] let LTT plot calculation be skipped, as it takes most time
[x] only measure correlation at the end
[ ] run on Peregrine

richelbilderbeek commented 7 years ago

let LTT plot calculation be skipped, as it takes most time is not as easily skipped as expected...

richelbilderbeek commented 7 years ago

Done that...

richelbilderbeek commented 7 years ago

A profile:

x0: 0.5
p0: 0.5
q0: 0.5
se: 0.147228
sm: 0.681808
sc: 0.627311
sq: 0.302464
c: 0.875845
at: 0.0328572
seed: 123
popsize: 10000
end_time: 10
sk: 0.0372092
sv: 0.829297
outputfrequency: 8
histw: 50
histbinx: 0.1
histbinp: 0.1
histbinq: 0.1
outputfilename: defaultresults
ltt_plot_filename: defaultlttplot
is silent: 1
Branching on ecotype: 0
Branching on mating: 0
Sympatric speciation: 0

real    0m58.310s
user    0m57.464s
sys 0m0.732s
Flat profile:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total           
 time   seconds   seconds    calls   s/call   s/call  name    
 25.41      2.04     2.04 1406899174     0.00     0.00  kewe::gauss(double, double)
 19.30      3.59     1.55        2     0.78     0.78  kewe::add_edges(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > const&, boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>&, double)
 18.93      5.11     1.52   100000     0.00     0.00  kewe::calc_competition(unsigned int, std::vector<kewe::individual, std::allocator<kewe::individual> > const&, kewe::simulation_parameters const&)
  7.72      5.73     0.62 203399587     0.00     0.00  kewe::calc_attractiveness(kewe::individual const&, kewe::individual const&, kewe::simulation_parameters const&)
  7.22      6.31     0.58  6839610     0.00     0.00  kewe::pick_individual(std::vector<double, std::allocator<double> > const&, double, std::mersenne_twister_engine<unsigned long, 32ul, 624ul, 397ul, 31ul, 2567483615ul, 11ul, 4294967295ul, 7ul, 2636928640ul, 15ul, 4022730752ul, 18ul, 1812433253ul>&)

richelbilderbeek commented 7 years ago

Only run calc_edges at end, instead of also at t=0:

x0: 0.5
p0: 0.5
q0: 0.5
se: 0.350687
sm: 0.262926
sc: 0.323423
sq: 0.515248
c: 0.561363
at: 0.0262375
seed: 123
popsize: 10000
end_time: 10
sk: 0.85377
sv: 0.426482
outputfrequency: 0
histw: 50
histbinx: 0.1
histbinp: 0.1
histbinq: 0.1
outputfilename: defaultresults
ltt_plot_filename: defaultlttplot
is silent: 1
Branching on ecotype: 0
Branching on mating: 0
Sympatric speciation: 0

real    0m51.406s
user    0m50.356s
sys 0m0.696s
Flat profile:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total           
 time   seconds   seconds    calls   s/call   s/call  name    
 32.24      2.65     2.65 1202170854     0.00     0.00  kewe::gauss(double, double)
 23.97      4.62     1.97        1     1.97     1.97  kewe::add_edges(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > const&, boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>&, double)
 17.76      6.08     1.46   100000     0.00     0.00  kewe::calc_competition(unsigned int, std::vector<kewe::individual, std::allocator<kewe::individual> > const&, kewe::simulation_parameters const&)
  6.20      6.59     0.51                             std::vector<boost::detail::adj_list_gen<boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>, boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>::config::stored_vertex, std::allocator<boost::detail::adj_list_gen<boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>, boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>::config::stored_vertex> >::_M_default_append(unsigned long)
  5.47      7.04     0.45 101035427     0.00     0.00  kewe::calc_attractiveness(kewe::individual const&, kewe::individual const&, kewe::simulation_parameters const&)

richelbilderbeek commented 7 years ago

I could

[x] write a class called gausser that should be a speedier version of gauss, as it is close to a look-up-table
[x] improve add_edges by adding a color to the vertices

richelbilderbeek commented 7 years ago

add_edges_impl_1:

real    7m42.369s
user    3m22.068s
sys 0m21.272s
Flat profile:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total           
 time   seconds   seconds    calls  ms/call  ms/call  name    
 44.60     20.81    20.81     1000    20.81    20.81  kewe::add_edges_impl_1(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > const&, boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>&, double)
 18.69     29.53     8.72                             void std::vector<boost::detail::stored_edge_iter<unsigned long, std::_List_iterator<boost::list_edge<unsigned long, boost::no_property> >, boost::no_property>, std::allocator<boost::detail::stored_edge_iter<unsigned long, std::_List_iterator<boost::list_edge<unsigned long, boost::no_property> >, boost::no_property> > >::_M_emplace_back_aux<boost::detail::stored_edge_iter<unsigned long, std::_List_iterator<boost::list_edge<unsigned long, boost::no_property> >, boost::no_property> const&>(boost::detail::stored_edge_iter<unsigned long, std::_List_iterator<boost::list_edge<unsigned long, boost::no_property> >, boost::no_property> const&)
 10.50     34.43     4.90 3002183234     0.00     0.00  kewe::gauss(double, double)
  5.89     37.18     2.75 1000591617     0.00     0.00  kewe::calc_attractiveness(kewe::individual const&, kewe::individual const&, kewe::simulation_parameters const&)
  4.95     39.49     2.31     1000     2.31     2.33  void boost::detail::depth_first_visit_impl<boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>, boost::detail::components_recorder<boost::iterator_property_map<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int> > >, boost::vec_adj_list_vertex_id_map<std::string, unsigned long>, int, int&> >, boost::shared_array_property_map<boost::default_color_type, boost::vec_adj_list_vertex_id_map<std::string, unsigned long> >, boost::detail::nontruth2>(boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS> const&, boost::graph_traits<boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS> >::vertex_descriptor, boost::detail::components_recorder<boost::iterator_property_map<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int> > >, boost::vec_adj_list_vertex_id_map<std::string, unsigned long>, int, int&> >&, boost::shared_array_property_map<boost::default_color_type, boost::vec_adj_list_vertex_id_map<std::string, unsigned long> >, boost::detail::nontruth2)

richelbilderbeek commented 7 years ago

add_edges_impl_2:

real    3m43.391s
user    3m7.104s
sys 0m20.440s
Flat profile:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total           
 time   seconds   seconds    calls  ms/call  ms/call  name    
 45.84     20.27    20.27     1000    20.27    20.27  kewe::add_edges_impl_2(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > const&, boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>&, double)
 18.34     28.38     8.11                             void std::vector<boost::detail::stored_edge_iter<unsigned long, std::_List_iterator<boost::list_edge<unsigned long, boost::no_property> >, boost::no_property>, std::allocator<boost::detail::stored_edge_iter<unsigned long, std::_List_iterator<boost::list_edge<unsigned long, boost::no_property> >, boost::no_property> > >::_M_emplace_back_aux<boost::detail::stored_edge_iter<unsigned long, std::_List_iterator<boost::list_edge<unsigned long, boost::no_property> >, boost::no_property> const&>(boost::detail::stored_edge_iter<unsigned long, std::_List_iterator<boost::list_edge<unsigned long, boost::no_property> >, boost::no_property> const&)
  9.00     32.36     3.98 3002183234     0.00     0.00  kewe::gauss(double, double)
  6.67     35.31     2.95 1000591617     0.00     0.00  kewe::calc_attractiveness(kewe::individual const&, kewe::individual const&, kewe::simulation_parameters const&)
  5.72     37.84     2.53     1000     2.53     2.54  void boost::detail::depth_first_visit_impl<boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>, boost::detail::components_recorder<boost::iterator_property_map<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int> > >, boost::vec_adj_list_vertex_id_map<std::string, unsigned long>, int, int&> >, boost::shared_array_property_map<boost::default_color_type, boost::vec_adj_list_vertex_id_map<std::string, unsigned long> >, boost::detail::nontruth2>(boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS> const&, boost::graph_traits<boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS> >::vertex_descriptor, boost::detail::components_recorder<boost::iterator_property_map<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int> > >, boost::vec_adj_list_vertex_id_map<std::string, unsigned long>, int, int&> >&, boost::shared_array_property_map<boost::default_color_type, boost::vec_adj_list_vertex_id_map<std::string, unsigned long> >, boost::detail::nontruth2)

Note: crappy implementation, indeed did nothing :-)

richelbilderbeek commented 7 years ago

With correct implementation:

real    2m13.835s
user    1m55.296s
sys 0m2.276s
Flat profile:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total           
 time   seconds   seconds    calls  ms/call  ms/call  name    
 29.10      3.51     3.51 3002183234     0.00     0.00  kewe::gauss(double, double)
 18.49      5.74     2.23 1000591617     0.00     0.00  kewe::calc_attractiveness(kewe::individual const&, kewe::individual const&, kewe::simulation_parameters const&)
 11.86      7.17     1.43     1000     1.43     5.99  kewe::calc_attractivenesses(std::vector<kewe::individual, std::allocator<kewe::individual> > const&, kewe::simulation_parameters const&)
 11.69      8.58     1.41  1000000     0.00     0.00  kewe::calc_competition(unsigned int, std::vector<kewe::individual, std::allocator<kewe::individual> > const&, kewe::simulation_parameters const&)
  9.04      9.67     1.09                             std::vector<boost::detail::adj_list_gen<boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>, boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>::config::stored_vertex, std::allocator<boost::detail::adj_list_gen<boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>, boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>::config::stored_vertex> >::_M_default_append(unsigned long)

richelbilderbeek commented 7 years ago

Latest profile:

real    0m35.332s
user    0m35.212s
sys 0m0.008s
Flat profile:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total           
 time   seconds   seconds    calls  ms/call  ms/call  name    
 55.73      1.80     1.80 1000367626     0.00     0.00  kewe::gauss(double, double)
 41.18      3.13     1.33   100000     0.01     0.03  kewe::calc_competition(unsigned int, std::vector<kewe::individual, std::allocator<kewe::individual> > const&, kewe::simulation_parameters const&)
  1.55      3.18     0.05   267642     0.00     0.00  kewe::pick_individual(std::vector<double, std::allocator<double> > const&, double, std::mersenne_twister_engine<unsigned long, 32ul, 624ul, 397ul, 31ul, 2567483615ul, 11ul, 4294967295ul, 7ul, 2636928640ul, 15ul, 4022730752ul, 18ul, 1812433253ul>&)
  0.62      3.20     0.02  1540369     0.00     0.00  double std::generate_canonical<double, 53ul, std::mersenne_twister_engine<unsigned long, 32ul, 624ul, 397ul, 31ul, 2567483615ul, 11ul, 4294967295ul, 7ul, 2636928640ul, 15ul, 4022730752ul, 18ul, 1812433253ul> >(std::mersenne_twister_engine<unsigned long, 32ul, 624ul, 397ul, 31ul, 2567483615ul, 11ul, 4294967295ul, 7ul, 2636928640ul, 15ul, 4022730752ul, 18ul, 1812433253ul>&)
  0.62      3.22     0.02       10     2.00   321.82  kewe::create_next_generation(kewe::simulation_parameters const&, std::vector<kewe::individual, std::allocator<kewe::individual> > const&, std::mersenne_twister_engine<unsigned long, 32ul, 624ul, 397ul, 31ul, 2567483615ul, 11ul, 4294967295ul, 7ul, 2636928640ul, 15ul, 4022730752ul, 18ul, 1812433253ul>&)

richelbilderbeek commented 7 years ago

First experiment is set up:

[s2260050@peregrine pbdmms]$ sbatch kewe_job_random 
Submitted batch job 10406834

It is even running:

[s2260050@peregrine pbdmms]$ cat *.out

The following have been reloaded with a version change:
  1) GLib/2.48.0-foss-2016a => GLib/2.47.5-foss-2016a
  2) gettext/0.19.7-foss-2016a => gettext/0.19.6-foss-2016a

x0: 0.5
p0: 0.5
q0: 0.5
se: 0.766739
sm: 0.173904
sc: 0.950975
sq: 0.531784
c: 0.67188
at: 0.0654884
seed: 123
popsize: 10000
end_time: 10
sk: 0.0505633
sv: 0.828116
outputfrequency: 0
histw: 50
histbinx: 0.1
histbinp: 0.1
histbinq: 0.1
outputfilename: defaultresults
ltt_plot_filename: defaultlttplot
is silent: 1

It is even in the queue:

[s2260050@peregrine pbdmms]$ squeue -u $USER
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
          10406834     nodes kewe_ran s2260050  R       0:14      1 pg-node005

richelbilderbeek commented 7 years ago

Ouch...

/var/spool/slurmd/job10406834/slurm_script: line 10: 139682 Killed                  ./kewe_peregrine --random
slurmstepd: error: Exceeded step memory limit at some point.

richelbilderbeek commented 7 years ago

10 GB should be enough?

richelbilderbeek commented 7 years ago

First successful run:

[s2260050@peregrine pbdmms]$ squeue -u $USER
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
[s2260050@peregrine pbdmms]$ cat *.out

The following have been reloaded with a version change:
  1) GLib/2.48.0-foss-2016a => GLib/2.47.5-foss-2016a
  2) gettext/0.19.7-foss-2016a => gettext/0.19.6-foss-2016a

x0: 0.5
p0: 0.5
q0: 0.5
se: 0.990887
sm: 0.369646
sc: 0.779792
sq: 0.180828
c: 0.727318
at: 0.0407095
seed: 123
popsize: 10000
end_time: 10
sk: 0.67166
sv: 0.245884
outputfrequency: 0
histw: 50
histbinx: 0.1
histbinp: 0.1
histbinq: 0.1
outputfilename: defaultresults
ltt_plot_filename: defaultlttplot
is silent: 1
Branching on ecotype: 0
Branching on mating: 0
Sympatric speciation: 0

###############################################################################
Peregrine Cluster
Job 10406841 for user 's2260050'
Finished at: Fri Dec 23 17:25:05 CET 2016

Job details:
============

Name                : kewe_random
User                : s2260050
Partition           : nodes
Nodes               : pg-node054
Cores               : 1
State               : RUNNING
Submit              : 2016-12-23T17:22:58
Start               : 2016-12-23T17:24:20
End                 : --
Reserved walltime   : 01:00:00
Used walltime       : 00:00:46
Used CPU time       : 00:00:40 (efficiency: 87.60%)
% User (Computation): 95.48%
% System (I/O)      :  4.52%
Mem reserved        : 10G/node
Max Mem used        : 5.11M (pg-node054)
Max Disk Write      : 92.16K (pg-node054)
Max Disk Read       : 2.11M (pg-node054)

Acknowledgements:
=================

Please see this page if you want to acknowledge Peregrine in your publications:

https://redmine.hpc.rug.nl/redmine/projects/peregrine/wiki/ScientificOutput

################################################################################

richelbilderbeek commented 7 years ago

Sure, only 10 generations, need 1K more, so I predict this will take 1000 minutes per run!

So I will need some extra speed. Or just be :see_no_evil: ?

Lumphie commented 7 years ago

Try some with 3k (Lyke's worked with 3k) or 1k (Sander's worked with 1k) individuals. Should be faster.

richelbilderbeek commented 7 years ago

Thanks. I will if I cannot shortcut the gauss function.

richelbilderbeek commented 7 years ago

gauss is called 1M times in the profile run, taking 50% of all time. So, if I create some gaussers that each use 4000 values, it would (1) not be noticable in the profile (2) maybe worth it.

richelbilderbeek commented 7 years ago

Use t_end = 5000, pop_size = 1000: predict 50 minutes

richelbilderbeek commented 7 years ago

OK, here goes:

[s2260050@peregrine pbdmms]$ squeue -u $USER
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
          10406871     nodes kewe_ran s2260050 PD       0:00      1 (Resources)
          10406872     nodes kewe_ran s2260050 PD       0:00      1 (Priority)

richelbilderbeek commented 7 years ago

gausser is only thrice as fast:

sum_gauss: 3.13309e+07
sum_gausser: 3.1327e+07
n_millis_gauss: 3251
n_millis_gausser: 1104

Assume my code takes 600 seconds
300 seconds are used by gauss
this will be 100 seconds by gausser, leaving 400 seconds
this makes my code only (400-600)/600=33% faster
that is not impressive

Thus I abandon this approach make code clean again.

richelbilderbeek commented 7 years ago

Okidokie, submitted 999 jobs :crossed_fingers: luckily on @Lumphie's account :sunglasses:

richelbilderbeek commented 7 years ago

@Lumphie : jobs are done. Due to the bug described at #160 , we can only grep for bimodal distributions in the sexual traits.

OTOH, one can also just use the sado sim now.

Lumphie commented 7 years ago

Can you still see mating type branching? That's the most interesting one anyway.

richelbilderbeek commented 7 years ago

Yes, on can see those, with something like

egrep -iR "mating type branching: 1" --include=*.log

but that has to be done on Peregrine from your account.

richelbilderbeek commented 7 years ago

Close this Issue unless @Lumphie reopens it. I do not see why he would, be well, you never know :-)

richelbilderbeek / pbdmms

SES: brute force branching parameter settings #149