ropensci / rnoaa

R interface to many NOAA data APIs
https://docs.ropensci.org/rnoaa
Other
328 stars 85 forks source link

error downloading data using ncdc function #391

Closed SaraAV closed 3 years ago

SaraAV commented 3 years ago

I am using ncdc function to download rainfall data for one station

start_date <- "1895-01-01"
end_date <- "1895-12-31"
datatype <- "PRCP"
siteNumber <- "GHCND:USC00023498"
data <- ncdc(datasetid='GHCND', stationid=siteNumber[j], datatypeid=datatype, startdate = start_date, enddate = end_date)$data

I have checked and this station has data until 1895-07-31, but the output is only until 1895-01-25:

> dput(data)
structure(list(date = c("1895-01-01T00:00:00", "1895-01-02T00:00:00", 
"1895-01-03T00:00:00", "1895-01-04T00:00:00", "1895-01-05T00:00:00", 
"1895-01-06T00:00:00", "1895-01-07T00:00:00", "1895-01-08T00:00:00", 
"1895-01-09T00:00:00", "1895-01-10T00:00:00", "1895-01-11T00:00:00", 
"1895-01-12T00:00:00", "1895-01-13T00:00:00", "1895-01-14T00:00:00", 
"1895-01-15T00:00:00", "1895-01-16T00:00:00", "1895-01-17T00:00:00", 
"1895-01-18T00:00:00", "1895-01-19T00:00:00", "1895-01-20T00:00:00", 
"1895-01-21T00:00:00", "1895-01-22T00:00:00", "1895-01-23T00:00:00", 
"1895-01-24T00:00:00", "1895-01-25T00:00:00"), datatype = c("PRCP", 
"PRCP", "PRCP", "PRCP", "PRCP", "PRCP", "PRCP", "PRCP", "PRCP", 
"PRCP", "PRCP", "PRCP", "PRCP", "PRCP", "PRCP", "PRCP", "PRCP", 
"PRCP", "PRCP", "PRCP", "PRCP", "PRCP", "PRCP", "PRCP", "PRCP"
), station = c("GHCND:USC00023498", "GHCND:USC00023498", "GHCND:USC00023498", 
"GHCND:USC00023498", "GHCND:USC00023498", "GHCND:USC00023498", 
"GHCND:USC00023498", "GHCND:USC00023498", "GHCND:USC00023498", 
"GHCND:USC00023498", "GHCND:USC00023498", "GHCND:USC00023498", 
"GHCND:USC00023498", "GHCND:USC00023498", "GHCND:USC00023498", 
"GHCND:USC00023498", "GHCND:USC00023498", "GHCND:USC00023498", 
"GHCND:USC00023498", "GHCND:USC00023498", "GHCND:USC00023498", 
"GHCND:USC00023498", "GHCND:USC00023498", "GHCND:USC00023498", 
"GHCND:USC00023498"), value = c(0L, 0L, 0L, 0L, 0L, 28L, 56L, 
0L, 0L, 0L, 0L, 0L, 0L, 0L, 94L, 76L, 102L, 0L, 0L, 137L, 0L, 
0L, 0L, 0L, 0L), fl_m = c("", "", "", "", "", "", "", "", "", 
"", "", "", "", "", "", "", "", "", "", "", "", "", "", "", ""
), fl_q = c("", "", "", "", "", "", "", "", "", "", "", "", "", 
"", "", "", "", "", "", "", "", "", "", "", ""), fl_so = c("0", 
"0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", 
"0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"), fl_t = c("", 
"", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", 
"", "", "", "", "", "", "", "")), row.names = c(NA, -25L), class = c("tbl_df", 
"tbl", "data.frame"))

I have also tried with other years and I have the same issue, only data for the first month is downloaded.

Is there any bug in the function? (I used this function on other occasions and was working properly.

sckott commented 3 years ago

thanks for your question @SaraAV

Try the limit parameter in ncdc() https://docs.ropensci.org/rnoaa/reference/ncdc.html#arguments

sckott commented 3 years ago

You can see details of pagination in the NOAA documentation https://www.ncdc.noaa.gov/cdo-web/webservices/v2

SaraAV commented 3 years ago

Thank you sckott!

sckott commented 3 years ago

Did the limit parameter fix your issue?

SaraAV commented 3 years ago

Yes, it was an issue with the limit parameter!

sckott commented 3 years ago

great!