search

crosenth / medirect

multiprocessed ncbi edirect
GNU General Public License v3.0
5 stars 3 forks source link

HTTP Error 429: Too Many Requests #5

Closed crosenth closed 5 years ago

crosenth commented 5 years ago

It looks like ncbi is actually throttling more than 3 requests per second. I am thinking of creating a new feature argument --intervial that will space out efetch arguments to 3 per/sec

crosenth commented 5 years ago

screenshot from 2018-12-12 10-19-14

crosenth commented 5 years ago

Fixed by forcing a 1 second pause in everyone request to ensure reqs/sec limits are not exceeded

crosenth commented 5 years ago

https://github.com/crosenth/medirect/commit/9bd281bcd2e39662a0a8ae2bc97322c172b587dd