search

BDI-pathogens / OpenABM-Covid19

OpenABM-Covid19: an agent-based model for modelling the spread of SARS-CoV-2 (coronavirus) and control interventions for the Covid-19 epidemic
GNU General Public License v3.0
114 stars 76 forks source link

Memory consumption #235

Open RobRomijnders opened 1 year ago

RobRomijnders commented 1 year ago

Hi all, thanks for implementing this API. It has enabled great research already.

I am running into memory problems. Running the simulator with 1 million users, it's intended use-case, my 60 gb memory run out in under 40 steps. Changing the 'days_of_interactions' parameter hasn't changed that.

Am I doing something wrong? What typical memory consumption do you observe?

Thanks in advance, Rob

p-robot commented 1 year ago

Hi Rob, Great to hear that OpenABM has been a useful tool!

Regarding your question, a simple run of the model with 1 million individuals should definitely not exhaust 60Gb of memory. The example in examples/example_101.py with 1M users runs without any hiccups on my 2016 MBP, which has 8Gb of physical memory: 

import COVID19.model as abm
model = abm.Model( params = { "n_total" : 1000000, "end_time": 50 } )
model.run()

Would it be possible for you to please provide a minimal example of where you're coming up against problems with memory? It will help us diagnose the issue.

Best regards, Will

RobRomijnders commented 1 year ago

Hi Will,

Thanks for the help. I digged a bit further and realized the memory leak was in the Python-bindings with SWIG. I'll make a pull-request soon to fix this.

Indeed, after the fix 100k users consumes less than 5gb and 1M users consumes less than 10gb.

p-robot commented 1 year ago

Hi Rob, That's good to hear that you've found a remedy in the interim. That'd be fantastic if you had a PR for the issue you found - many thanks for contributing.
Will