hmgu-itg / peakplotter

A tool to find and annotate signals in next-generation association studies
6 stars 3 forks source link

Wrap a retry and sleep function on the GET request to Ensembl REST API #4

Open youngchanpark opened 3 years ago

youngchanpark commented 3 years ago

Since we usually run loads of peakplotter run in parallel, a lot of the runs fail due to rate-limit imposed by Ensembl.

Ensembl REST API Rate limits https://github.com/Ensembl/ensembl-rest/wiki/Rate-Limits

One IP address is allowed 55,000 requests per hour. We can extract the time remaining for the rate limit reset in the HTTP header. So it would be useful to have a sleep function to wait until the rate limit resets and start running again.

We should also have a retry wrapper so the function wouldn't just sleep indefinitely (doubt this would happen but still).