search

ropensci / rnoaa

R interface to many NOAA data APIs
https://docs.ropensci.org/rnoaa
Other
330 stars 84 forks source link

Nonsensical return values from ncdc(datasetid = 'GHCND') #378

Closed amcdavid closed 3 years ago

amcdavid commented 3 years ago

When I get GHCND data, the temperature values do not seem remotely correct (and disagree with the CSV that I manually download from https://www.ncdc.noaa.gov/cdo-web/search, which looks plausible).

ibrary(rnoaa)
# Rochester, NY (not Mars)
foo = ncdc(datasetid = 'GHCND', stationid = 'GHCND:USW00014768', startdate = '2020-01-01', enddate = '2020-01-31', datatypeid = c('TMIN', "TMAX", 'TAVG'))
foo$data
# A tibble: 25 x 8
   date                datatype station           value fl_m  fl_q  fl_so fl_t  
                                        
 1 2020-01-01T00:00:00 TAVG     GHCND:USW00014768    -6 "H"   ""    S     ""    
 2 2020-01-01T00:00:00 TMAX     GHCND:USW00014768     6 ""    ""    W     "2400"
 3 2020-01-01T00:00:00 TMIN     GHCND:USW00014768   -21 ""    ""    W     "2400"
 4 2020-01-02T00:00:00 TAVG     GHCND:USW00014768    26 "H"   ""    S     ""    
 5 2020-01-02T00:00:00 TMAX     GHCND:USW00014768    89 ""    ""    W     "2400"
 6 2020-01-02T00:00:00 TMIN     GHCND:USW00014768   -27 ""    ""    W     "2400"
 7 2020-01-03T00:00:00 TAVG     GHCND:USW00014768    67 "H"   ""    S     ""    
 8 2020-01-03T00:00:00 TMAX     GHCND:USW00014768    83 ""    ""    W     "2400"
 9 2020-01-03T00:00:00 TMIN     GHCND:USW00014768    44 ""    ""    W     "2400"
10 2020-01-04T00:00:00 TAVG     GHCND:USW00014768    26 "H"   ""    S     ""

FWIW, I get the same apparent nonsense with curl on the command line, so this is probably not an issue with rnoaa or parsing the json but rather with the NCDC REST API itself. I wanted to confirm I wasn't going crazy before figuring out who/how to escalate on their end.

amcdavid commented 3 years ago

Looking more carefully at the units when I set add_units = TRUE, I now realize this is returning, as advertised, ten times the temperature in degrees celcius, so the values are both reasonable (and correct in light of the results in the .csv which are in fahrenheit). The integer-type should have been a warning to me...

sckott commented 3 years ago

FYI, add_units is not TRUE by default because it's still in development - there's a lot of NOAA datasets, with A LOT of different variables, many of which are not documented very well. We're doing our best, and can always use help from folks telling us if any variables are wrong - and filling in more information - the files where we keep units info are the files starting with units- here https://github.com/ropensci/rnoaa/tree/master/R