search

openforcefield / openff-qcsubmit

Automated tools for submitting molecules to QCFractal
https://openff-qcsubmit.readthedocs.io/en/latest/index.html
MIT License
26 stars 4 forks source link

Removed chunked saves for entry submission; added performance hack for single point task submission #186

Closed dotsdl closed 2 years ago

dotsdl commented 2 years ago

Description

This removes chunked calls to dataset.save as entries (molecules) are added to a given dataset as part of submission, as these are unnecessary and only serve to slow submission as the collection object grows with additional entries added.

We also added in a performance hack for single point Datasets only, in which we only call collection.save on the first call for a given spec addition and then again after all tasks are generated. This saves repeated movement of the collection object, which is unchanged as tasks are generated over the course of the loop.

Requires a new release of QCFractal beyond the current 0.15.7 release, in particular MolSSI/QCFractal#705.

Status

jthorton commented 2 years ago

@dotsdl thanks for adding this speed up! Looks like we need a new version of qcfractal to test on do we know when we will have a realese?

dotsdl commented 2 years ago

@jthorton hopefully today!

dotsdl commented 2 years ago

Once we get #189 in, I can finish this out.

jthorton commented 2 years ago

@dotsdl merged the other PR feel free to merge this one once tests pass

codecov[bot] commented 2 years ago

Codecov Report

Merging #186 (57e458b) into main (4fcbd79) will decrease coverage by 0.03%. The diff coverage is 88.88%.

dotsdl commented 2 years ago

Thanks @jthorton! Merging!