search

kblin / ncbi-acc-download

Download files from NCBI Entrez by accession
Apache License 2.0
111 stars 8 forks source link

NCBI Entrez returned error code 429, are ID(s) [...] valid? #20

Open orenmn opened 4 years ago

orenmn commented 4 years ago

I have tried to run ncbi-acc-download --verbose --recursive BLEV00000000.1 multiple times, but got a similar error message each time (only different id ranges):

Failed to download file with id BLEV01000101,BLEV01000102,BLEV01000103,BLEV01000104,BLEV01000105,BLEV01000106,BLEV01000107,BLEV01000108,BLEV01000109,BLEV01000110 from NCBI
NCBI Entrez returned error code 429, are ID(s) BLEV01000101,BLEV01000102,BLEV01000103,BLEV01000104,BLEV01000105,BLEV01000106,BLEV01000107,BLEV01000108,BLEV01000109,BLEV01000110 valid?

https://www.ncbi.nlm.nih.gov/nuccore/BLEV01000101 and https://www.ncbi.nlm.nih.gov/nuccore/BLEV01000102 (for example) seem valid to me, but maybe I am missing something..

kblin commented 4 years ago

You're running too many requests per time slice, which is why you're getting the 429 status. You can either wait a bit and continue, or get an NCBI API key and use the git version of ncbi-acc-download with the --api-key parameter.

orenmn commented 4 years ago

Interesting. I used the command on two other WGS master records, and it worked every time, then when I used it on BLEV00000000.1, it failed every time. This experimentation was several hours ago.

Now I only tried ncbi-acc-download --verbose --recursive BLEV00000000.1 without any other downloads before that, and indeed it worked.

Thank you!! (I am not closing the issue in case you wish to change the error message or something, but you definitely solved my problem)

kblin commented 4 years ago

I have the impression that the NCBI clamped down on how many requests you could send before running into yay 429 status recently. I used to almost never hit it, and in the last month I kept running into it frequently.