Trying to add a large number of entries can cause a timeout on the server. This adds automatic batching (with progress bars) so that this doesn't happen.
Also adds similar functionality to submit().
In the future, submit() (and maybe add_entries()) should move to an asynchronous operation, where an internal job is created, which can be viewed/polled with some future-like object.
Description
Trying to add a large number of entries can cause a timeout on the server. This adds automatic batching (with progress bars) so that this doesn't happen.
Also adds similar functionality to
submit().In the future,
submit()(and maybeadd_entries()) should move to an asynchronous operation, where an internal job is created, which can be viewed/polled with some future-like object.See https://github.com/openmm/spice-dataset/issues/82 for some motivation, although that's not the only one.
Changelog description
Add automatic batching in ds.add_entries() and ds.submit()
Status