I am trying to pull large results (69 pages of 1000 results) from the API, I sometimes have to retry 5 times to not get a 502 (Bad Gateway) error from nginx.
Is it caused by the query being too large and python timing-out on nginx?
Is there a recommend number of results to pull in a page?
What command did you use?
Something like
curl https://mriqc.nimh.nih.gov/api/v1/bold?max_results=1000&where={"bids_meta.MagneticFieldStrength":3,"bids_meta.MultibandAccelerationFactor":4,"bids_meta.RepetitionTime":{"$lt":1.6,"$gt":1.4},"spacing_x":{"$lte":2.2,"$gte":1.8},"spacing_y":{"$lte":2.2,"$gte":1.8},"spacing_z":{"$lte":2.2,"$gte":1.8},"size_t":{"$gte":100}}&page=9
### What version of the software are you running?
na
### How are you running this software?
Local installation ("bare-metal")
### Is your data BIDS valid?
Yes
### Are you reusing any previously computed results?
No
### Please copy and paste any relevant log output.
_No response_
### Additional information / screenshots
_No response_
What happened?
I am trying to pull large results (69 pages of 1000 results) from the API, I sometimes have to retry 5 times to not get a 502 (Bad Gateway) error from nginx. Is it caused by the query being too large and python timing-out on nginx? Is there a recommend number of results to pull in a page?
What command did you use?