search

ropensci / rnoaa

R interface to many NOAA data APIs
https://docs.ropensci.org/rnoaa
Other
330 stars 84 forks source link

download failed for isd #410

Open TianyaImpression opened 2 years ago

TianyaImpression commented 2 years ago

R version 4.1.2 (2021-11-01) -- "Bird Hippie"

for (i in 35:nrow(x_cn2)) { for (j in 1:length(year)) { res1 <- isd(usaf=x_cn2[i,1], wban='99999', year= year[j]) write.csv(res1, file = paste0("./csv/", x_cn2[i,1], "-", year[j], ".csv")) } } I have successfully downloaded more than 700 ISD data, when I continue to download, it always prompts "Error: download failed for https://www1.ncdc.noaa.gov/pub/data/noaa/2001/561320-99999-2001.gz”

I have entered NOAA_KEY

image

I would like to know if there is a limit on the amount of data that rnoaa can download? Is there a way to download more?

Thank you for your reply!

djhocking commented 2 years ago

I am not sure offhand but can look into it. Does the error always occur at the same number of records (or same total resulting folder size) or is it failing on particular records? If it's related to the number of requests you can submit in a certain amount of time, then I've seen hacks where you can put a wait/pause function in the for loop to get around an API query rate.

TianyaImpression commented 2 years ago

I am not sure offhand but can look into it. Does the error always occur at the same number of records (or same total resulting folder size) or is it failing on particular records? If it's related to the number of requests you can submit in a certain amount of time, then I've seen hacks where you can put a wait/pause function in the for loop to get around an API query rate.

I found that it may be a problem with some sites or some years. I can continue to download after skipping some failed sites or years. Thank you for your reply.