materialsproject / fireworks

The Fireworks Workflow Management Repo.
https://materialsproject.github.io/fireworks
Other
351 stars 184 forks source link

Excessive DB read ops #463

Closed janosh closed 2 years ago

janosh commented 2 years ago

I tried out MongoDBs new pay-as-you-go Serverless instance with fireworks after running out of space on an M0 cluster.

For some reason, fireworks seems to be performing excessive read ops.

Screen Shot 2021-11-11 at 10 18 07

@computron Any ideas why that is? I used qlaunch rapidfire --nlaunches 100 once and lpad webgui and lpad get_fws a few times during this time and can't understand how that adds up to 175 million reads. Can this be made more efficient?

@tschaume Maybe you have some advice on how to profile where the DB requests are mostly coming from?