Parallelize generation of `webviz_storage`?

equinor / webviz-config

Make Dash applications from a user-friendly config file :book: :snake:

https://github.com/orgs/equinor/projects/24

MIT License

54 stars 38 forks source link

Parallelize generation of `webviz_storage`? #50

Open anders-kiaer opened 5 years ago

HansKallekleiv commented 5 years ago

Relevant code: https://github.com/equinor/webviz-config/blob/7b812455fbff62794e18176b85c0bbb626370673/webviz_config/webviz_store.py#L143-L163

Is it possible to parallellize this for each func or perhaps each argtuple using multiprocessing?

anders-kiaer commented 5 years ago

Build of webviz_store currently utilizes caching for "cache decorated functions". If to be continued, this issue should probably be seen together with #67.

Is store building currently typically CPU bound (=multiprocessing) or I/O bound (=threading)?