search

GeoscienceAustralia / eqrm

Automatically exported from code.google.com/p/eqrm
Other
5 stars 4 forks source link

demo's are slow (eqrm_core/demo$ python setdata_ScenGM.py) #35

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Robbie has a laptop install of EQRM that is taking 4 mins to execute python 
setdata_ScenGM.py and 24 mins to run on a desktop install.

timing info
______________________________________

pc 

pmgo@pmgo-desktop:~/Models/EQRM/eqrm_core/demo$ python setdata_ScenGM.py 
time_pre_site_loop_fraction 0.000586029067042
event_loop_time (excluding file saving) 0:08:31.920000 hr:min:sec
On node 0, pmgo-desktop clock (processor) time taken overall 0:08:32.920000 
hr:min:sec.
On node 0, pmgo-desktop wall time taken overall 0:24:20.636670 hr:min:sec.
wall_time_taken_overall_seconds = 1460.63666987

laptop
time_pre_site_loop_fraction 0.00370581527936
event_loop_time (excluding file saving) 0:02:55.400000 hr:min:sec
On node 0, felix-laptop clock (processor) time taken overall 0:02:57.630000 
hr:min:sec.
On node 0, felix-laptop wall time taken overall 0:06:33.452604 hr:min:sec.
wall_time_taken_overall_seconds = 393.452604055
felix@felix-laptop:~/Models/EQRM/eqrm_core/trunk/demo$

pc
pmgo@pmgo-desktop:~/Models/EQRM/eqrm_core$ python test_all.py
Testing path /home/pmgo/Models/EQRM/eqrm_core/eqrm_code/..:
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
....................
----------------------------------------------------------------------
Ran 420 tests in 16.587s

OK
pmgo@pmgo-desktop:~/Models/EQRM/eqrm_core$ 

laptop
felix@felix-laptop:~/Models/EQRM/eqrm_core/trunk$ python test_all.py 

Testing path /home/felix/Models/EQRM/eqrm_core/trunk/eqrm_code/..:

................................................................................
................................................................................
................................................................................
................................................................................
.........................................................................
----------------------------------------------------------------------
Ran 393 tests in 6.145s
OK
felix@felix-laptop:~/Models/EQRM/eqrm_core/trunk$ 

________________________________________________________

I'm assuming this can be fixed if it doesn't save the arrays to file

Original issue reported on code.google.com by duncan.g...@gmail.com on 22 Mar 2012 at 11:02

GoogleCodeExporter commented 9 years ago

Original comment by b...@girorosso.com on 22 Mar 2012 at 11:24

GoogleCodeExporter commented 9 years ago 
Looking at the timing on rhe-compute1 for setdata_ScenGM.py:

time_pre_site_loop_fraction 0.744186046512
event_loop_time (excluding file saving) 0:00:00.430000 hr:min:sec
On node 0, rhe-compute1.ga.gov.au clock (processor) time taken overall 
0:00:00.450000 hr:min:sec.
On node 0, rhe-compute1.ga.gov.au wall time taken overall 0:00:00.888135 
hr:min:sec.
wall_time_taken_overall_seconds = 0.888135194778

Adding file_array = False to setdata_ScenGM.py:

time_pre_site_loop_fraction 0.892857142857
event_loop_time (excluding file saving) 0:00:00.280000 hr:min:sec
On node 0, rhe-compute1.ga.gov.au clock (processor) time taken overall 
0:00:00.290000 hr:min:sec.
On node 0, rhe-compute1.ga.gov.au wall time taken overall 0:00:00.438409 
hr:min:sec.
wall_time_taken_overall_seconds = 0.438408851624

Original comment by b...@girorosso.com on 22 Mar 2012 at 11:35

GoogleCodeExporter commented 9 years ago 
Some more timings for the demos

file_array = True
============================
============================
Doing  setdata_ProbHaz.py
time_pre_site_loop_fraction 0.553480475382
event_loop_time (excluding file saving) 0:00:05.890000 hr:min:sec
On node 0, rhe-compute1.ga.gov.au clock (processor) time taken overall 
0:00:05.900000 hr:min:sec.
On node 0, rhe-compute1.ga.gov.au wall time taken overall 0:00:08.080821 
hr:min:sec.
wall_time_taken_overall_seconds = 8.08082103729
============================
============================
Doing  setdata_ScenGM.py
time_pre_site_loop_fraction 0.659574468085
event_loop_time (excluding file saving) 0:00:00.470000 hr:min:sec
On node 0, rhe-compute1.ga.gov.au clock (processor) time taken overall 
0:00:00.510000 hr:min:sec.
On node 0, rhe-compute1.ga.gov.au wall time taken overall 0:00:00.701699 
hr:min:sec.
wall_time_taken_overall_seconds = 0.701699018478
============================
============================
Doing  setdata_ScenRisk.py
time_pre_site_loop_fraction 0.0658747300216
event_loop_time (excluding file saving) 0:00:09.260000 hr:min:sec
On node 0, rhe-compute1.ga.gov.au clock (processor) time taken overall 
0:00:09.740000 hr:min:sec.
On node 0, rhe-compute1.ga.gov.au wall time taken overall 0:00:11.411284 
hr:min:sec.
wall_time_taken_overall_seconds = 11.4112842083
============================
============================
Doing  setdata_ScenRisk2.py
time_pre_site_loop_fraction 0.0347826086957
event_loop_time (excluding file saving) 0:00:18.400000 hr:min:sec
On node 0, rhe-compute1.ga.gov.au clock (processor) time taken overall 
0:00:19.010000 hr:min:sec.
On node 0, rhe-compute1.ga.gov.au wall time taken overall 0:00:20.556379 
hr:min:sec.
wall_time_taken_overall_seconds = 20.5563788414
============================
============================
Doing  setdata_ProbHaz.py
time_pre_site_loop_fraction 0.556325823224
event_loop_time (excluding file saving) 0:00:05.770000 hr:min:sec
On node 0, rhe-compute1.ga.gov.au clock (processor) time taken overall 
0:00:05.780000 hr:min:sec.
On node 0, rhe-compute1.ga.gov.au wall time taken overall 0:00:07.959820 
hr:min:sec.
wall_time_taken_overall_seconds = 7.95982003212
============================
============================
Doing  setdata_ProbRisk.py
time_pre_site_loop_fraction 0.492469879518
event_loop_time (excluding file saving) 0:00:06.640000 hr:min:sec
On node 0, rhe-compute1.ga.gov.au clock (processor) time taken overall 
0:00:06.990000 hr:min:sec.
On node 0, rhe-compute1.ga.gov.au wall time taken overall 0:00:09.023833 
hr:min:sec.
wall_time_taken_overall_seconds = 9.02383303642

file_array = False
============================
============================
Doing  setdata_ProbHaz.py
time_pre_site_loop_fraction 0.892045454545
event_loop_time (excluding file saving) 0:00:03.520000 hr:min:sec
On node 0, rhe-compute1.ga.gov.au clock (processor) time taken overall 
0:00:03.520000 hr:min:sec.
On node 0, rhe-compute1.ga.gov.au wall time taken overall 0:00:03.790861 
hr:min:sec.
wall_time_taken_overall_seconds = 3.79086112976
============================
============================
Doing  setdata_ScenGM.py
time_pre_site_loop_fraction 0.92
event_loop_time (excluding file saving) 0:00:00.250000 hr:min:sec
On node 0, rhe-compute1.ga.gov.au clock (processor) time taken overall 
0:00:00.250000 hr:min:sec.
On node 0, rhe-compute1.ga.gov.au wall time taken overall 0:00:00.300790 
hr:min:sec.
wall_time_taken_overall_seconds = 0.300790071487
============================
============================
Doing  setdata_ScenRisk.py
time_pre_site_loop_fraction 0.0992509363296
event_loop_time (excluding file saving) 0:00:05.340000 hr:min:sec
On node 0, rhe-compute1.ga.gov.au clock (processor) time taken overall 
0:00:05.730000 hr:min:sec.
On node 0, rhe-compute1.ga.gov.au wall time taken overall 0:00:05.829611 
hr:min:sec.
wall_time_taken_overall_seconds = 5.829611063
============================
============================
Doing  setdata_ScenRisk2.py
time_pre_site_loop_fraction 0.0349462365591
event_loop_time (excluding file saving) 0:00:14.880000 hr:min:sec
On node 0, rhe-compute1.ga.gov.au clock (processor) time taken overall 
0:00:15.400000 hr:min:sec.
On node 0, rhe-compute1.ga.gov.au wall time taken overall 0:00:15.496367 
hr:min:sec.
wall_time_taken_overall_seconds = 15.4963669777
============================
============================
Doing  setdata_ProbHaz.py
time_pre_site_loop_fraction 0.89367816092
event_loop_time (excluding file saving) 0:00:03.480000 hr:min:sec
On node 0, rhe-compute1.ga.gov.au clock (processor) time taken overall 
0:00:03.480000 hr:min:sec.
On node 0, rhe-compute1.ga.gov.au wall time taken overall 0:00:03.559463 
hr:min:sec.
wall_time_taken_overall_seconds = 3.55946302414
============================
============================
Doing  setdata_ProbRisk.py
time_pre_site_loop_fraction 0.732265446224
event_loop_time (excluding file saving) 0:00:04.370000 hr:min:sec
On node 0, rhe-compute1.ga.gov.au clock (processor) time taken overall 
0:00:04.630000 hr:min:sec.
On node 0, rhe-compute1.ga.gov.au wall time taken overall 0:00:04.704458 
hr:min:sec.
wall_time_taken_overall_seconds = 4.70445799828

Original comment by b...@girorosso.com on 22 Mar 2012 at 11:54

GoogleCodeExporter commented 9 years ago 
Revision 1016 sets the parameter file_array = False for demos. Timing 
improvements are as per the comments above.

Original comment by b...@girorosso.com on 23 Mar 2012 at 12:10