Profiling scoring code on local setup to identify hot spots for speed/memory improvement

Khoale1096 commented 4 years ago

A sub-issue of this issue: https://github.com/reichlab/forecast-repository/issues/246

Tasks to be done:

[x] Get Local machine set up to run all the tests on Postgres.
[x] Load a snapshot of the production database and make sure you can calculate all scores locally. You'll need to be able to run the local dev web server. Scores will require installing Redis so that RQ will work. (We'll need to factor in the number of workers into profiling. Waiting 3 hours for a run with just one worker would be painful.)
[ ] Once you have all that local dev infrastructure working (a non-trivial task in itself!) then you can start instrumenting the scoring code to figure out where time is spent. Profiling memory might be more challenging, so I'd say start with time. We can talk approaches, but I have a number of logging statements, and typically I just figure out time deltas using the datetimes that logging prints.
[ ] Once you have some numbers, have a discussion to talk about possible scoring changes to make, which we will do and then verify they actually improved things!

mzorn-58 commented 4 years ago

Khoa- Given the fact that the plan is to move scoring out of Zoltar within the next month or two, Nick does not want you too spend too much more time on this. If you can finish this up in a short time (<5hours) or add code that might help, then go for it. Otherwise you can stop working on this.

Khoale1096 commented 3 years ago

Closing since scoring is now moved to covidHubUtils

reichlab / forecast-repository

Profiling scoring code on local setup to identify hot spots for speed/memory improvement #248