USEPA / ElectricityLCI

Creative Commons Zero v1.0 Universal
24 stars 10 forks source link

using joblib memory class to improve performance for duplicate function calls #173

Open bl-young opened 2 years ago

bl-young commented 2 years ago

importing of some modules, particularly those that use stewi data results in objects being stored in memory (example)

Moving those function calls inside functions may result in needing to call the function multiple times.

joblib Memory class, or something similar, could be helpful for caching those function calls to improve performance https://joblib.readthedocs.io/en/latest/memory.html

m-jamieson commented 1 year ago

You happen to try this at all? Looking at the example, there is a lot that goes on inside that module with almost nothing inside functions. I think the most efficient thing to do in this place is to put everything inside a function and cache that result. I guess I would need to take a closer look to see what variables need to be exposed to the modules calling it.

bl-young commented 1 year ago

I think I recall when I looked into it a bit that it doesn't work so well for caching results of function calls that are data frames. But it's been a while. Feel free to close this it may not be useful.