biocore / redbiom

Sample search by metadata and features
Other
44 stars 20 forks source link

Fetching many samples is slower than batching externally #98

Closed wasade closed 1 year ago

wasade commented 4 years ago

Fetching a reasonable number of samples (e.g., >10k) takes a longer than expected. The runtime is better if the input is split into smaller batches of samples with individual calls. This suggests there is some type of performance scaling issue on fetch.