search

ropensci / rnoaa

R interface to many NOAA data APIs
https://docs.ropensci.org/rnoaa
Other
330 stars 84 forks source link

ncdc_datasets doesn't use datasetid #401

Open muschellij2 opened 3 years ago

muschellij2 commented 3 years ago

It would seem that the datasetid is not used in ncdc_datasets. The URL creation here https://github.com/ropensci/rnoaa/blob/master/R/ncdc_datasets.r#L52 are not passed as ncdc_GET is called without URL/datasetid parameters: https://github.com/ropensci/rnoaa/blob/master/R/ncdc_datasets.r#L69

Also there is no filtration as the data is done with the data: https://github.com/ropensci/rnoaa/blob/master/R/ncdc_datasets.r#L73

library(rnoaa)
library(purrr)
ncdc_datasets()
#> $meta
#> $meta$offset
#> [1] 1
#> 
#> $meta$count
#> [1] 11
#> 
#> $meta$limit
#> [1] 25
#> 
#> 
#> $data
#>                     uid    mindate    maxdate                        name
#> 1  gov.noaa.ncdc:C00861 1763-01-01 2021-09-08             Daily Summaries
#> 2  gov.noaa.ncdc:C00946 1763-01-01 2021-09-01 Global Summary of the Month
#> 3  gov.noaa.ncdc:C00947 1763-01-01 2021-01-01  Global Summary of the Year
#> 4  gov.noaa.ncdc:C00345 1991-06-05 2021-09-07    Weather Radar (Level II)
#> 5  gov.noaa.ncdc:C00708 1994-05-20 2021-09-07   Weather Radar (Level III)
#> 6  gov.noaa.ncdc:C00821 2010-01-01 2010-01-01     Normals Annual/Seasonal
#> 7  gov.noaa.ncdc:C00823 2010-01-01 2010-12-31               Normals Daily
#> 8  gov.noaa.ncdc:C00824 2010-01-01 2010-12-31              Normals Hourly
#> 9  gov.noaa.ncdc:C00822 2010-01-01 2010-12-01             Normals Monthly
#> 10 gov.noaa.ncdc:C00505 1970-05-12 2014-01-01     Precipitation 15 Minute
#> 11 gov.noaa.ncdc:C00313 1900-01-01 2014-01-01        Precipitation Hourly
#>    datacoverage         id
#> 1          1.00      GHCND
#> 2          1.00       GSOM
#> 3          1.00       GSOY
#> 4          0.95    NEXRAD2
#> 5          0.95    NEXRAD3
#> 6          1.00 NORMAL_ANN
#> 7          1.00 NORMAL_DLY
#> 8          1.00 NORMAL_HLY
#> 9          1.00 NORMAL_MLY
#> 10         0.25  PRECIP_15
#> 11         1.00 PRECIP_HLY
#> 
#> attr(,"class")
#> [1] "ncdc_datasets"
ncdc_datasets("GHCND")
#> $meta
#> NULL
#> 
#> $data
#>   metadata.resultset.offset metadata.resultset.count metadata.resultset.limit
#> 1                         1                       11                       25
#>            results.uid results.mindate results.maxdate    results.name
#> 1 gov.noaa.ncdc:C00861      1763-01-01      2021-09-08 Daily Summaries
#>   results.datacoverage results.id        results.uid.1 results.mindate.1
#> 1                    1      GHCND gov.noaa.ncdc:C00946        1763-01-01
#>   results.maxdate.1              results.name.1 results.datacoverage.1
#> 1        2021-09-01 Global Summary of the Month                      1
#>   results.id.1        results.uid.2 results.mindate.2 results.maxdate.2
#> 1         GSOM gov.noaa.ncdc:C00947        1763-01-01        2021-01-01
#>               results.name.2 results.datacoverage.2 results.id.2
#> 1 Global Summary of the Year                      1         GSOY
#>          results.uid.3 results.mindate.3 results.maxdate.3
#> 1 gov.noaa.ncdc:C00345        1991-06-05        2021-09-07
#>             results.name.3 results.datacoverage.3 results.id.3
#> 1 Weather Radar (Level II)                   0.95      NEXRAD2
#>          results.uid.4 results.mindate.4 results.maxdate.4
#> 1 gov.noaa.ncdc:C00708        1994-05-20        2021-09-07
#>              results.name.4 results.datacoverage.4 results.id.4
#> 1 Weather Radar (Level III)                   0.95      NEXRAD3
#>          results.uid.5 results.mindate.5 results.maxdate.5
#> 1 gov.noaa.ncdc:C00821        2010-01-01        2010-01-01
#>            results.name.5 results.datacoverage.5 results.id.5
#> 1 Normals Annual/Seasonal                      1   NORMAL_ANN
#>          results.uid.6 results.mindate.6 results.maxdate.6 results.name.6
#> 1 gov.noaa.ncdc:C00823        2010-01-01        2010-12-31  Normals Daily
#>   results.datacoverage.6 results.id.6        results.uid.7 results.mindate.7
#> 1                      1   NORMAL_DLY gov.noaa.ncdc:C00824        2010-01-01
#>   results.maxdate.7 results.name.7 results.datacoverage.7 results.id.7
#> 1        2010-12-31 Normals Hourly                      1   NORMAL_HLY
#>          results.uid.8 results.mindate.8 results.maxdate.8  results.name.8
#> 1 gov.noaa.ncdc:C00822        2010-01-01        2010-12-01 Normals Monthly
#>   results.datacoverage.8 results.id.8        results.uid.9 results.mindate.9
#> 1                      1   NORMAL_MLY gov.noaa.ncdc:C00505        1970-05-12
#>   results.maxdate.9          results.name.9 results.datacoverage.9 results.id.9
#> 1        2014-01-01 Precipitation 15 Minute                   0.25    PRECIP_15
#>         results.uid.10 results.mindate.10 results.maxdate.10
#> 1 gov.noaa.ncdc:C00313         1900-01-01         2014-01-01
#>        results.name.10 results.datacoverage.10 results.id.10
#> 1 Precipitation Hourly                       1    PRECIP_HLY
#> 
#> attr(,"class")
#> [1] "ncdc_datasets"
ncdc_datasets() %>% 
  pluck("data") %>% 
  dplyr::filter(id %in% "GHCND")
#>                    uid    mindate    maxdate            name datacoverage    id
#> 1 gov.noaa.ncdc:C00861 1763-01-01 2021-09-08 Daily Summaries            1 GHCND

Created on 2021-09-10 by the reprex package (v2.0.0)

Session info ``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.1.0 (2021-05-18) #> os Debian GNU/Linux 10 (buster) #> system x86_64, linux-gnu #> ui X11 #> language (EN) #> collate C.UTF-8 #> ctype C.UTF-8 #> tz Etc/UTC #> date 2021-09-10 #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date lib source #> assertthat 0.2.1 2019-03-21 [2] CRAN (R 4.1.0) #> cli 3.0.1 2021-07-17 [1] CRAN (R 4.1.0) #> colorspace 2.0-1 2021-05-04 [2] CRAN (R 4.1.0) #> crayon 1.4.1 2021-02-08 [2] CRAN (R 4.1.0) #> crul 1.1.0 2021-02-15 [1] CRAN (R 4.1.0) #> curl 4.3.2 2021-06-23 [1] CRAN (R 4.1.0) #> DBI 1.1.1 2021-01-15 [2] CRAN (R 4.1.0) #> digest 0.6.27 2020-10-24 [2] CRAN (R 4.1.0) #> dplyr 1.0.7 2021-06-18 [2] CRAN (R 4.1.0) #> ellipsis 0.3.2 2021-04-29 [2] CRAN (R 4.1.0) #> evaluate 0.14 2019-05-28 [2] CRAN (R 4.1.0) #> fansi 0.5.0 2021-05-25 [2] CRAN (R 4.1.0) #> fs 1.5.0 2020-07-31 [2] CRAN (R 4.1.0) #> generics 0.1.0 2020-10-31 [2] CRAN (R 4.1.0) #> ggplot2 3.3.4 2021-06-16 [2] CRAN (R 4.1.0) #> glue 1.4.2 2020-08-27 [2] CRAN (R 4.1.0) #> gridExtra 2.3 2017-09-09 [2] CRAN (R 4.1.0) #> gtable 0.3.0 2019-03-25 [2] CRAN (R 4.1.0) #> highr 0.9 2021-04-16 [2] CRAN (R 4.1.0) #> hoardr 0.5.2 2018-12-02 [1] CRAN (R 4.1.0) #> htmltools 0.5.1.1 2021-01-22 [2] CRAN (R 4.1.0) #> httpcode 0.3.0 2020-04-10 [1] CRAN (R 4.1.0) #> jsonlite 1.7.2 2020-12-09 [2] CRAN (R 4.1.0) #> knitr 1.33 2021-04-24 [2] CRAN (R 4.1.0) #> lifecycle 1.0.0 2021-02-15 [2] CRAN (R 4.1.0) #> lubridate 1.7.10 2021-02-26 [2] CRAN (R 4.1.0) #> magrittr 2.0.1 2020-11-17 [2] CRAN (R 4.1.0) #> munsell 0.5.0 2018-06-12 [2] CRAN (R 4.1.0) #> pillar 1.6.1 2021-05-16 [2] CRAN (R 4.1.0) #> pkgconfig 2.0.3 2019-09-22 [2] CRAN (R 4.1.0) #> purrr * 0.3.4 2020-04-17 [2] CRAN (R 4.1.0) #> R6 2.5.1 2021-08-19 [1] CRAN (R 4.1.0) #> rappdirs 0.3.3 2021-01-31 [2] CRAN (R 4.1.0) #> Rcpp 1.0.6 2021-01-15 [2] CRAN (R 4.1.0) #> reprex 2.0.0 2021-04-02 [2] CRAN (R 4.1.0) #> rlang 0.4.11 2021-04-30 [2] CRAN (R 4.1.0) #> rmarkdown 2.9 2021-06-15 [2] CRAN (R 4.1.0) #> rnoaa * 1.3.4 2021-05-19 [1] CRAN (R 4.1.0) #> rstudioapi 0.13 2020-11-12 [2] CRAN (R 4.1.0) #> scales 1.1.1 2020-05-11 [2] CRAN (R 4.1.0) #> sessioninfo 1.1.1 2018-11-05 [2] CRAN (R 4.1.0) #> stringi 1.7.3 2021-07-16 [1] CRAN (R 4.1.0) #> stringr 1.4.0 2019-02-10 [2] CRAN (R 4.1.0) #> tibble 3.1.2 2021-05-16 [2] CRAN (R 4.1.0) #> tidyr 1.1.3 2021-03-03 [2] CRAN (R 4.1.0) #> tidyselect 1.1.1 2021-04-30 [2] CRAN (R 4.1.0) #> triebeard 0.3.0 2016-08-04 [1] CRAN (R 4.1.0) #> urltools 1.7.3 2019-04-14 [1] CRAN (R 4.1.0) #> utf8 1.2.1 2021-03-12 [2] CRAN (R 4.1.0) #> vctrs 0.3.8 2021-04-29 [2] CRAN (R 4.1.0) #> withr 2.4.2 2021-04-18 [2] CRAN (R 4.1.0) #> xfun 0.24 2021-06-15 [2] CRAN (R 4.1.0) #> XML 3.99-0.6 2021-03-16 [2] CRAN (R 4.1.0) #> xml2 1.3.2 2020-04-23 [2] CRAN (R 4.1.0) #> yaml 2.2.1 2020-02-01 [2] CRAN (R 4.1.0) #> #> [1] /home/jupyter/.R/library #> [2] /usr/local/lib/R/site-library #> [3] /usr/lib/R/site-library #> [4] /usr/lib/R/library ```
djhocking commented 1 year ago

Sorry I was on leave and you had a workaround. Then the NCDC API was changed completely so I am just circling back to this. It will be a bit of a slog to get through the ncdc updates but I try to fix this as part of the adjustment to the new API structure. Thanks for sharing the issue.