Closed richelbilderbeek closed 7 years ago
let LTT plot calculation be skipped, as it takes most time
is not as easily skipped as expected...
Done that...
A profile:
x0: 0.5
p0: 0.5
q0: 0.5
se: 0.147228
sm: 0.681808
sc: 0.627311
sq: 0.302464
c: 0.875845
at: 0.0328572
seed: 123
popsize: 10000
end_time: 10
sk: 0.0372092
sv: 0.829297
outputfrequency: 8
histw: 50
histbinx: 0.1
histbinp: 0.1
histbinq: 0.1
outputfilename: defaultresults
ltt_plot_filename: defaultlttplot
is silent: 1
Branching on ecotype: 0
Branching on mating: 0
Sympatric speciation: 0
real 0m58.310s
user 0m57.464s
sys 0m0.732s
Flat profile:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls s/call s/call name
25.41 2.04 2.04 1406899174 0.00 0.00 kewe::gauss(double, double)
19.30 3.59 1.55 2 0.78 0.78 kewe::add_edges(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > const&, boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>&, double)
18.93 5.11 1.52 100000 0.00 0.00 kewe::calc_competition(unsigned int, std::vector<kewe::individual, std::allocator<kewe::individual> > const&, kewe::simulation_parameters const&)
7.72 5.73 0.62 203399587 0.00 0.00 kewe::calc_attractiveness(kewe::individual const&, kewe::individual const&, kewe::simulation_parameters const&)
7.22 6.31 0.58 6839610 0.00 0.00 kewe::pick_individual(std::vector<double, std::allocator<double> > const&, double, std::mersenne_twister_engine<unsigned long, 32ul, 624ul, 397ul, 31ul, 2567483615ul, 11ul, 4294967295ul, 7ul, 2636928640ul, 15ul, 4022730752ul, 18ul, 1812433253ul>&)
Only run calc_edges
at end, instead of also at t=0:
x0: 0.5
p0: 0.5
q0: 0.5
se: 0.350687
sm: 0.262926
sc: 0.323423
sq: 0.515248
c: 0.561363
at: 0.0262375
seed: 123
popsize: 10000
end_time: 10
sk: 0.85377
sv: 0.426482
outputfrequency: 0
histw: 50
histbinx: 0.1
histbinp: 0.1
histbinq: 0.1
outputfilename: defaultresults
ltt_plot_filename: defaultlttplot
is silent: 1
Branching on ecotype: 0
Branching on mating: 0
Sympatric speciation: 0
real 0m51.406s
user 0m50.356s
sys 0m0.696s
Flat profile:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls s/call s/call name
32.24 2.65 2.65 1202170854 0.00 0.00 kewe::gauss(double, double)
23.97 4.62 1.97 1 1.97 1.97 kewe::add_edges(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > const&, boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>&, double)
17.76 6.08 1.46 100000 0.00 0.00 kewe::calc_competition(unsigned int, std::vector<kewe::individual, std::allocator<kewe::individual> > const&, kewe::simulation_parameters const&)
6.20 6.59 0.51 std::vector<boost::detail::adj_list_gen<boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>, boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>::config::stored_vertex, std::allocator<boost::detail::adj_list_gen<boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>, boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>::config::stored_vertex> >::_M_default_append(unsigned long)
5.47 7.04 0.45 101035427 0.00 0.00 kewe::calc_attractiveness(kewe::individual const&, kewe::individual const&, kewe::simulation_parameters const&)
I could
gausser
that should be a speedier version of gauss
, as it is close to a look-up-tableadd_edges
by adding a color to the verticesadd_edges_impl_1
:
real 7m42.369s
user 3m22.068s
sys 0m21.272s
Flat profile:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls ms/call ms/call name
44.60 20.81 20.81 1000 20.81 20.81 kewe::add_edges_impl_1(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > const&, boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>&, double)
18.69 29.53 8.72 void std::vector<boost::detail::stored_edge_iter<unsigned long, std::_List_iterator<boost::list_edge<unsigned long, boost::no_property> >, boost::no_property>, std::allocator<boost::detail::stored_edge_iter<unsigned long, std::_List_iterator<boost::list_edge<unsigned long, boost::no_property> >, boost::no_property> > >::_M_emplace_back_aux<boost::detail::stored_edge_iter<unsigned long, std::_List_iterator<boost::list_edge<unsigned long, boost::no_property> >, boost::no_property> const&>(boost::detail::stored_edge_iter<unsigned long, std::_List_iterator<boost::list_edge<unsigned long, boost::no_property> >, boost::no_property> const&)
10.50 34.43 4.90 3002183234 0.00 0.00 kewe::gauss(double, double)
5.89 37.18 2.75 1000591617 0.00 0.00 kewe::calc_attractiveness(kewe::individual const&, kewe::individual const&, kewe::simulation_parameters const&)
4.95 39.49 2.31 1000 2.31 2.33 void boost::detail::depth_first_visit_impl<boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>, boost::detail::components_recorder<boost::iterator_property_map<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int> > >, boost::vec_adj_list_vertex_id_map<std::string, unsigned long>, int, int&> >, boost::shared_array_property_map<boost::default_color_type, boost::vec_adj_list_vertex_id_map<std::string, unsigned long> >, boost::detail::nontruth2>(boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS> const&, boost::graph_traits<boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS> >::vertex_descriptor, boost::detail::components_recorder<boost::iterator_property_map<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int> > >, boost::vec_adj_list_vertex_id_map<std::string, unsigned long>, int, int&> >&, boost::shared_array_property_map<boost::default_color_type, boost::vec_adj_list_vertex_id_map<std::string, unsigned long> >, boost::detail::nontruth2)
add_edges_impl_2
:
real 3m43.391s
user 3m7.104s
sys 0m20.440s
Flat profile:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls ms/call ms/call name
45.84 20.27 20.27 1000 20.27 20.27 kewe::add_edges_impl_2(std::vector<std::vector<double, std::allocator<double> >, std::allocator<std::vector<double, std::allocator<double> > > > const&, boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>&, double)
18.34 28.38 8.11 void std::vector<boost::detail::stored_edge_iter<unsigned long, std::_List_iterator<boost::list_edge<unsigned long, boost::no_property> >, boost::no_property>, std::allocator<boost::detail::stored_edge_iter<unsigned long, std::_List_iterator<boost::list_edge<unsigned long, boost::no_property> >, boost::no_property> > >::_M_emplace_back_aux<boost::detail::stored_edge_iter<unsigned long, std::_List_iterator<boost::list_edge<unsigned long, boost::no_property> >, boost::no_property> const&>(boost::detail::stored_edge_iter<unsigned long, std::_List_iterator<boost::list_edge<unsigned long, boost::no_property> >, boost::no_property> const&)
9.00 32.36 3.98 3002183234 0.00 0.00 kewe::gauss(double, double)
6.67 35.31 2.95 1000591617 0.00 0.00 kewe::calc_attractiveness(kewe::individual const&, kewe::individual const&, kewe::simulation_parameters const&)
5.72 37.84 2.53 1000 2.53 2.54 void boost::detail::depth_first_visit_impl<boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>, boost::detail::components_recorder<boost::iterator_property_map<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int> > >, boost::vec_adj_list_vertex_id_map<std::string, unsigned long>, int, int&> >, boost::shared_array_property_map<boost::default_color_type, boost::vec_adj_list_vertex_id_map<std::string, unsigned long> >, boost::detail::nontruth2>(boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS> const&, boost::graph_traits<boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS> >::vertex_descriptor, boost::detail::components_recorder<boost::iterator_property_map<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int> > >, boost::vec_adj_list_vertex_id_map<std::string, unsigned long>, int, int&> >&, boost::shared_array_property_map<boost::default_color_type, boost::vec_adj_list_vertex_id_map<std::string, unsigned long> >, boost::detail::nontruth2)
Note: crappy implementation, indeed did nothing :-)
With correct implementation:
real 2m13.835s
user 1m55.296s
sys 0m2.276s
Flat profile:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls ms/call ms/call name
29.10 3.51 3.51 3002183234 0.00 0.00 kewe::gauss(double, double)
18.49 5.74 2.23 1000591617 0.00 0.00 kewe::calc_attractiveness(kewe::individual const&, kewe::individual const&, kewe::simulation_parameters const&)
11.86 7.17 1.43 1000 1.43 5.99 kewe::calc_attractivenesses(std::vector<kewe::individual, std::allocator<kewe::individual> > const&, kewe::simulation_parameters const&)
11.69 8.58 1.41 1000000 0.00 0.00 kewe::calc_competition(unsigned int, std::vector<kewe::individual, std::allocator<kewe::individual> > const&, kewe::simulation_parameters const&)
9.04 9.67 1.09 std::vector<boost::detail::adj_list_gen<boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>, boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>::config::stored_vertex, std::allocator<boost::detail::adj_list_gen<boost::adjacency_list<boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>, boost::vecS, boost::vecS, boost::undirectedS, std::string, boost::no_property, boost::no_property, boost::listS>::config::stored_vertex> >::_M_default_append(unsigned long)
Latest profile:
real 0m35.332s
user 0m35.212s
sys 0m0.008s
Flat profile:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls ms/call ms/call name
55.73 1.80 1.80 1000367626 0.00 0.00 kewe::gauss(double, double)
41.18 3.13 1.33 100000 0.01 0.03 kewe::calc_competition(unsigned int, std::vector<kewe::individual, std::allocator<kewe::individual> > const&, kewe::simulation_parameters const&)
1.55 3.18 0.05 267642 0.00 0.00 kewe::pick_individual(std::vector<double, std::allocator<double> > const&, double, std::mersenne_twister_engine<unsigned long, 32ul, 624ul, 397ul, 31ul, 2567483615ul, 11ul, 4294967295ul, 7ul, 2636928640ul, 15ul, 4022730752ul, 18ul, 1812433253ul>&)
0.62 3.20 0.02 1540369 0.00 0.00 double std::generate_canonical<double, 53ul, std::mersenne_twister_engine<unsigned long, 32ul, 624ul, 397ul, 31ul, 2567483615ul, 11ul, 4294967295ul, 7ul, 2636928640ul, 15ul, 4022730752ul, 18ul, 1812433253ul> >(std::mersenne_twister_engine<unsigned long, 32ul, 624ul, 397ul, 31ul, 2567483615ul, 11ul, 4294967295ul, 7ul, 2636928640ul, 15ul, 4022730752ul, 18ul, 1812433253ul>&)
0.62 3.22 0.02 10 2.00 321.82 kewe::create_next_generation(kewe::simulation_parameters const&, std::vector<kewe::individual, std::allocator<kewe::individual> > const&, std::mersenne_twister_engine<unsigned long, 32ul, 624ul, 397ul, 31ul, 2567483615ul, 11ul, 4294967295ul, 7ul, 2636928640ul, 15ul, 4022730752ul, 18ul, 1812433253ul>&)
First experiment is set up:
[s2260050@peregrine pbdmms]$ sbatch kewe_job_random
Submitted batch job 10406834
It is even running:
[s2260050@peregrine pbdmms]$ cat *.out
The following have been reloaded with a version change:
1) GLib/2.48.0-foss-2016a => GLib/2.47.5-foss-2016a
2) gettext/0.19.7-foss-2016a => gettext/0.19.6-foss-2016a
x0: 0.5
p0: 0.5
q0: 0.5
se: 0.766739
sm: 0.173904
sc: 0.950975
sq: 0.531784
c: 0.67188
at: 0.0654884
seed: 123
popsize: 10000
end_time: 10
sk: 0.0505633
sv: 0.828116
outputfrequency: 0
histw: 50
histbinx: 0.1
histbinp: 0.1
histbinq: 0.1
outputfilename: defaultresults
ltt_plot_filename: defaultlttplot
is silent: 1
It is even in the queue:
[s2260050@peregrine pbdmms]$ squeue -u $USER
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
10406834 nodes kewe_ran s2260050 R 0:14 1 pg-node005
Ouch...
/var/spool/slurmd/job10406834/slurm_script: line 10: 139682 Killed ./kewe_peregrine --random
slurmstepd: error: Exceeded step memory limit at some point.
10 GB should be enough?
First successful run:
[s2260050@peregrine pbdmms]$ squeue -u $USER
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
[s2260050@peregrine pbdmms]$ cat *.out
The following have been reloaded with a version change:
1) GLib/2.48.0-foss-2016a => GLib/2.47.5-foss-2016a
2) gettext/0.19.7-foss-2016a => gettext/0.19.6-foss-2016a
x0: 0.5
p0: 0.5
q0: 0.5
se: 0.990887
sm: 0.369646
sc: 0.779792
sq: 0.180828
c: 0.727318
at: 0.0407095
seed: 123
popsize: 10000
end_time: 10
sk: 0.67166
sv: 0.245884
outputfrequency: 0
histw: 50
histbinx: 0.1
histbinp: 0.1
histbinq: 0.1
outputfilename: defaultresults
ltt_plot_filename: defaultlttplot
is silent: 1
Branching on ecotype: 0
Branching on mating: 0
Sympatric speciation: 0
###############################################################################
Peregrine Cluster
Job 10406841 for user 's2260050'
Finished at: Fri Dec 23 17:25:05 CET 2016
Job details:
============
Name : kewe_random
User : s2260050
Partition : nodes
Nodes : pg-node054
Cores : 1
State : RUNNING
Submit : 2016-12-23T17:22:58
Start : 2016-12-23T17:24:20
End : --
Reserved walltime : 01:00:00
Used walltime : 00:00:46
Used CPU time : 00:00:40 (efficiency: 87.60%)
% User (Computation): 95.48%
% System (I/O) : 4.52%
Mem reserved : 10G/node
Max Mem used : 5.11M (pg-node054)
Max Disk Write : 92.16K (pg-node054)
Max Disk Read : 2.11M (pg-node054)
Acknowledgements:
=================
Please see this page if you want to acknowledge Peregrine in your publications:
https://redmine.hpc.rug.nl/redmine/projects/peregrine/wiki/ScientificOutput
################################################################################
Sure, only 10 generations, need 1K more, so I predict this will take 1000 minutes per run!
So I will need some extra speed. Or just be :see_no_evil: ?
Try some with 3k (Lyke's worked with 3k) or 1k (Sander's worked with 1k) individuals. Should be faster.
Thanks. I will if I cannot shortcut the gauss
function.
gauss
is called 1M times in the profile run, taking 50% of all time. So, if I create some gausser
s that each use 4000 values, it would (1) not be noticable in the profile (2) maybe worth it.
Use t_end = 5000, pop_size = 1000
: predict 50 minutes
OK, here goes:
[s2260050@peregrine pbdmms]$ squeue -u $USER
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
10406871 nodes kewe_ran s2260050 PD 0:00 1 (Resources)
10406872 nodes kewe_ran s2260050 PD 0:00 1 (Priority)
gausser
is only thrice as fast:
sum_gauss: 3.13309e+07
sum_gausser: 3.1327e+07
n_millis_gauss: 3251
n_millis_gausser: 1104
gauss
gausser
, leaving 400 secondsThus I abandon this approach make code clean again.
Okidokie, submitted 999 jobs :crossed_fingers: luckily on @Lumphie's account :sunglasses:
@Lumphie : jobs are done. Due to the bug described at #160 , we can only grep
for bimodal distributions in the sexual traits.
OTOH, one can also just use the sado
sim now.
Can you still see mating type branching? That's the most interesting one anyway.
Yes, on can see those, with something like
egrep -iR "mating type branching: 1" --include=*.log
but that has to be done on Peregrine from your account.
Close this Issue unless @Lumphie reopens it. I do not see why he would, be well, you never know :-)
parameters
objectsoperator<<
forparameters
parameters
, usingoperator<<