search

RSGInc / bca4abm

Benefit Cost Analysis for Travel Demand Models
http://rsginc.github.io/bca4abm/
Other
7 stars 5 forks source link

improve runtime #87

Closed bstabler closed 5 years ago

bstabler commented 5 years ago

it would be good to speed up the slow processors - most likely the od processor. We can test with the full Oregon Metro example data set and set of expressions.

bstabler commented 5 years ago

Here are runtimes for the full Oregon Metro example

INFO - activitysim.core.tracing - Time to execute run_model aggregate_demographics_processor : 2.42 seconds (0.0 minutes)
INFO - activitysim.core.tracing - Time to execute run_model aggregate_zone_processor : 1.826 seconds (0.0 minutes)
INFO - activitysim.core.tracing - Time to execute run_model aggregate_od_processor : 1596.351 seconds (26.6 minutes)
INFO - activitysim.core.tracing - Time to execute run_model link_daily_processor : 270.826 seconds (4.5 minutes)
INFO - activitysim.core.tracing - Time to execute run_model write_data_dictionary : 0.11 seconds (0.0 minutes)
INFO - activitysim.core.tracing - Time to execute run_model write_tables : 0.438 seconds (0.0 minutes)
INFO - activitysim.core.tracing - Time to execute run (6 models) : 1872.208 seconds (31.2 minutes)
INFO - activitysim.core.tracing - Time to execute all models : 1872.239 seconds (31.2 minutes)
bstabler commented 5 years ago

The cache_skims setting in aggregate_od.yaml lets you trade off RAM versus runtime. For example:

cache_skims: True Time to execute all models : 718.714 seconds (12.0 minutes) aggregate_od mem after assign_variables, memory_info: vms: 65.67 GB rss: 65.68 GB uss: 65.93 GB

cache_skims: False INFO - activitysim.core.tracing - Time to execute all models : 981.126 seconds (16.4 minutes) aggregate_od mem after assign_variables, memory_info: vms: 20.3 GB rss: 20.32 GB uss: 20.39 GB