ropensci / rnoaa

R interface to many NOAA data APIs
https://docs.ropensci.org/rnoaa
Other
330 stars 84 forks source link

odd values in ncdc() fxn #233

Closed sckott closed 5 years ago

sckott commented 7 years ago

getting some odd values (at least ones we can’t explain). For example, when I try to get TMIN for the weather station at Sea-Tac international airport:

ncdc(datasetid='GHCND', stationid="GHCND:USW00024233", datatypeid='TMIN', startdate = '2017-01-01', enddate = '2017-01-10')$data
#>                   date datatype           station value fl_m fl_q fl_so fl_t
#> 1  2017-01-01T00:00:00     TMIN GHCND:USW00024233   -21               W 2400
#> 2  2017-01-02T00:00:00     TMIN GHCND:USW00024233   -32               W 2400
#> 3  2017-01-03T00:00:00     TMIN GHCND:USW00024233   -60               W 2400
#> 4  2017-01-04T00:00:00     TMIN GHCND:USW00024233   -55               W 2400
#> 5  2017-01-05T00:00:00     TMIN GHCND:USW00024233   -60               W 2400
#> 6  2017-01-06T00:00:00     TMIN GHCND:USW00024233   -66               W 2400
#> 7  2017-01-07T00:00:00     TMIN GHCND:USW00024233   -16               W 2400
#> 8  2017-01-08T00:00:00     TMIN GHCND:USW00024233    17               W 2400
#> 9  2017-01-09T00:00:00     TMIN GHCND:USW00024233    11               W 2400
#> 10 2017-01-10T00:00:00     TMIN GHCND:USW00024233     0               W 2400

Do you have any idea how to interpret these numbers? A TMIN of -66 just seems odd J

andypicke commented 6 years ago

Also having a similar issue. Trying to get daily min/max temps, and getting crazy values. Tried converting from Kelvins, still didn't make sense. According to docs, should be in Fahrenheit or Celsius.

library(rnoaa)
library(dplyr)

x <- ncdc(datasetid='GHCND',stationid='GHCND:USC00084289',startdate="2017-06-01", enddate="2017-08-31", token=rnoaa_token)

xt <- x$data %>% filter(datatype %in% c('TMIN','TMAX'))
head(xt,10)

                  date datatype           station value fl_m fl_q fl_so fl_t
1  2017-06-01T00:00:00     TMAX GHCND:USC00084289   350               7 0700
2  2017-06-01T00:00:00     TMIN GHCND:USC00084289   189               7 0700
3  2017-06-02T00:00:00     TMAX GHCND:USC00084289   344               7 0700
4  2017-06-02T00:00:00     TMIN GHCND:USC00084289   217               7 0700
5  2017-06-03T00:00:00     TMAX GHCND:USC00084289   294               7 0700
6  2017-06-03T00:00:00     TMIN GHCND:USC00084289   200               7 0700
7  2017-06-04T00:00:00     TMAX GHCND:USC00084289   306               7 0700
8  2017-06-04T00:00:00     TMIN GHCND:USC00084289   206               7 0700
9  2017-06-05T00:00:00     TMAX GHCND:USC00084289   256               7 0700
10 2017-06-05T00:00:00     TMIN GHCND:USC00084289   217               7 0700

sessionInfo()
R version 3.4.3 (2017-11-30)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows Server 2008 R2 x64 (build 7601) Service Pack 1

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] bindrcpp_0.2 dplyr_0.7.4  rnoaa_0.7.0 

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.15     pillar_1.1.0     compiler_3.4.3   plyr_1.8.4       bindr_0.1        tools_3.4.3     
 [7] digest_0.6.14    memoise_1.1.0    lubridate_1.7.1  jsonlite_1.5     tibble_1.4.1     gtable_0.2.0    
[13] pkgconfig_2.0.1  rlang_0.1.6      curl_3.1         gridExtra_2.3    withr_2.1.1      httr_1.3.1      
[19] stringr_1.2.0    xml2_1.1.1       rappdirs_0.3.1   devtools_1.13.4  grid_3.4.3       glue_1.2.0      
[25] R6_2.2.2         XML_3.98-1.9     ggplot2_2.2.1    tidyr_0.7.2      purrr_0.2.4      hoardr_0.2.0    
[31] magrittr_1.5     scales_0.5.0     assertthat_0.2.0 colorspace_1.3-2 stringi_1.1.6    lazyeval_0.2.1  
[37] munsell_0.4.3   
sckott commented 6 years ago

thanks @andypicke will make sure to look into this now since at least 2 people have reported problems

andypicke commented 6 years ago

I looked at the source data for Daily Summaries at https://www.ncdc.noaa.gov/cdo-web/datasets . If you look at the data samples, the values seem correct in the pdf version (https://www1.ncdc.noaa.gov/pub/data/cdo/samples/GHCND_sample_pdf.pdf), but not the csv/excel/ascii versions (https://www1.ncdc.noaa.gov/pub/data/cdo/samples/GHCND_sample_ascii.dat). So it might be an issue with the data, not with rnoaa. I sent an email asking them about this, will update if I hear back.

sckott commented 6 years ago

perfect, thanks

andypicke commented 6 years ago

Heard back from NOAA on this. The daily Tmin and Tmax values are in tenths of degrees C.

sckott commented 6 years ago

awesome, thanks