search

AutomatedProcessImprovement / Simod

Simod is a tool for automated BPS model discovery
Apache License 2.0
40 stars 7 forks source link

Investigate long running times when using pooled resources that occurred during benchmarking for Simod 3.5.24 #123

Closed iharsuvorau closed 1 year ago

iharsuvorau commented 1 year ago

During benchmarking using differentiated resources, the Call Center event log processing took about half a day (4-6 hours). When using pooled resources, the same log took about 80 hours.

iharsuvorau commented 1 year ago

I can confirm the performance degradation for Academic Credentials (it's a relatively small log to work with) on my laptop:

AC_CRE differentiated: simod optimize --config_path config_differentiated.yml  1601.50s user 1307.30s system 556% cpu 8:42.95 total
AC_CRE pooled:         simod optimize --config_path config_pooled.yml          2352.18s user 1370.86s system 275% cpu 22:28.94 total
iharsuvorau commented 1 year ago

It seems the performance degradation comes from pix_framework.discovery.resource_model.discover_resource_model.

This is runtime for two functions from resource model optimizer run with pooled resources: 

optimizer._discover_resource_model: 20.360493183135986 s
optimizer._simulate_bps_model: 5.144507169723511 s

The same for differentiated resources:

optimizer._discover_resource_model: 0.4659249782562256 s
optimizer._simulate_bps_model: 5.323170185089111 s
iharsuvorau commented 1 year ago

See update in https://github.com/AutomatedProcessImprovement/pix-framework/issues/4