search

ropensci / weathercan

R package for downloading weather data from Environment and Climate Change Canada
https://docs.ropensci.org/weathercan
GNU General Public License v3.0
102 stars 29 forks source link

weather_dl() throws "Error: `guess_max` must be a positive integer" #119

Closed KevCaz closed 3 years ago

KevCaz commented 3 years ago

Hi @steffilazerte,

Expected Behavior

kam <- weather_dl(station_ids = 51423, 
                     start = "2016-01-01", end = "2016-02-15")

Should return a data frame with desired data.

Current Behavior

R> kam <- weather_dl(station_ids = 51423, 
                     start = "2016-01-01", end = "2016-02-15")   
Error: `guess_max` must be a positive integer

Steps to Reproduce (for bugs)

  1. Install latest version
  2. run first example from weather.R

Possible Solution

A quick investigation of mine showed that

https://github.com/ropensci/weathercan/blob/fe13093ebcb51c916c8f1bf011874d12d67081ac/R/weather.R#L659

now returns a NA which I think explains the error return by read_tsv(). When I changed this, then

https://github.com/ropensci/weathercan/blob/fe13093ebcb51c916c8f1bf011874d12d67081ac/R/weather.R#L661-L663

becomes a one colum data frame (instead of a 2 column one), so there might be more to this. Not sure why, I may investigate this further. Anyway, thanks for your job on this very useful package.

Context

Running some code that was previously working.

Your Environment

See below for my config, note that I was also able to reproduce this on another machine (MacOS 11.2.3 R 4.0.4).

``` sessionInfo() R version 4.0.4 (2021-02-15) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Debian GNU/Linux bullseye/sid Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.13.so locale: [1] LC_CTYPE=en_CA.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_CA.UTF-8 LC_COLLATE=en_CA.UTF-8 [5] LC_MONETARY=en_CA.UTF-8 LC_MESSAGES=en_CA.UTF-8 [7] LC_PAPER=en_CA.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_CA.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods [7] base other attached packages: [1] vcr_0.6.0 weathercan_0.5.0.9000 [3] rhub_1.1.1 pkgdown_1.6.1 [5] microbenchmark_1.4-7 goodpractice_1.0.2 [7] testthat_3.0.1 devtools_2.3.2 [9] usethis_2.0.0 loaded via a namespace (and not attached): [1] httr_1.4.2 pkgload_1.1.0 tidyr_1.1.3 [4] jsonlite_1.7.2 assertthat_0.2.1 xmlparsedata_1.0.5 [7] triebeard_0.3.0 urltools_1.7.3 yaml_2.2.1 [10] remotes_2.2.0 sessioninfo_1.1.1 pillar_1.6.0 [13] glue_1.4.2 uuid_0.1-4 digest_0.6.27 [16] rvest_1.0.0 clisymbols_1.2.0 pkgconfig_2.0.3 [19] rcmdcheck_1.3.3 httpcode_0.3.0 purrr_0.3.4 [22] processx_3.4.5 whisker_0.4 webmockr_0.7.4 [25] tibble_3.1.1 generics_0.1.0 ellipsis_0.3.1 [28] cachem_1.0.4 withr_2.4.1 lazyeval_0.2.2 [31] cli_2.5.0 magrittr_2.0.1 crayon_1.4.1 [34] memoise_2.0.0 ps_1.6.0 fs_1.5.0 [37] fansi_0.4.2 xml2_1.3.2 parsedate_1.2.1 [40] pkgbuild_1.2.0 praise_1.0.0 tools_4.0.4 [43] data.table_1.14.0 prettyunits_1.1.1 hms_1.0.0 [46] cyclocomp_1.1.0 lifecycle_1.0.0 stringr_1.4.0 [49] xopen_1.0.0 targets_0.4.1 callr_3.5.1 [52] rex_1.2.0 compiler_4.0.4 covr_3.5.1 [55] rlang_0.4.10 rstudioapi_0.13 rappdirs_0.3.3 [58] igraph_1.2.6 base64enc_0.1-3 codetools_0.2-18 [61] DBI_1.1.1 rematch_1.0.1 curl_4.3 [64] fauxpas_0.5.0 R6_2.5.0 lubridate_1.7.10 [67] dplyr_1.0.5 fastmap_1.1.0 utf8_1.1.4 [70] rprojroot_2.0.2 lintr_2.0.1 readr_1.4.0 [73] desc_1.2.0 stringi_1.5.3 crul_1.1.0 [76] whoami_1.3.0 Rcpp_1.0.6 vctrs_0.3.7 [79] tidyselect_1.1.0 R> devtools::session_info() ─ Session info ──────────────────────────────────────────────── setting value version R version 4.0.4 (2021-02-15) os Debian GNU/Linux bullseye/sid system x86_64, linux-gnu ui X11 language en_CA:en collate en_CA.UTF-8 ctype en_CA.UTF-8 tz America/Toronto date 2021-04-29 ─ Packages ──────────────────────────────────────────────────── ! package * version date lib assertthat 0.2.1 2019-03-21 [3] base64enc 0.1-3 2015-07-28 [3] cachem 1.0.4 2021-02-13 [1] callr 3.5.1 2020-10-13 [3] cli 2.5.0 2021-04-26 [1] clisymbols 1.2.0 2017-05-21 [3] codetools 0.2-18 2020-11-04 [4] covr 3.5.1 2020-09-16 [3] crayon 1.4.1 2021-02-08 [1] crul 1.1.0 2021-04-06 [1] curl 4.3 2019-12-02 [3] cyclocomp 1.1.0 2016-09-10 [1] data.table 1.14.0 2021-02-21 [1] DBI 1.1.1 2021-01-15 [3] desc 1.2.0 2018-05-01 [3] devtools * 2.3.2 2020-09-18 [3] digest 0.6.27 2020-10-24 [3] dplyr 1.0.5 2021-03-05 [1] ellipsis 0.3.1 2020-05-15 [3] fansi 0.4.2 2021-01-15 [3] fastmap 1.1.0 2021-01-25 [3] fauxpas 0.5.0 2020-04-13 [3] fs 1.5.0 2020-07-31 [1] generics 0.1.0 2020-10-31 [3] glue 1.4.2 2020-08-27 [3] goodpractice * 1.0.2 2018-05-02 [1] hms 1.0.0 2021-01-13 [3] httpcode 0.3.0 2020-04-10 [3] httr 1.4.2 2020-07-20 [3] igraph 1.2.6 2020-10-06 [3] jsonlite 1.7.2 2020-12-09 [1] lazyeval 0.2.2 2019-03-15 [3] lifecycle 1.0.0 2021-02-15 [1] lintr 2.0.1 2020-02-19 [1] lubridate 1.7.10 2021-02-26 [1] magrittr 2.0.1 2020-11-17 [3] memoise 2.0.0 2021-01-26 [3] microbenchmark * 1.4-7 2019-09-24 [1] parsedate 1.2.1 2021-04-20 [1] pillar 1.6.0 2021-04-13 [1] pkgbuild 1.2.0 2020-12-15 [3] pkgconfig 2.0.3 2019-09-22 [3] pkgdown * 1.6.1 2020-09-12 [1] pkgload 1.1.0 2020-05-29 [3] praise 1.0.0 2015-08-11 [3] prettyunits 1.1.1 2020-01-24 [3] processx 3.4.5 2020-11-30 [3] ps 1.6.0 2021-02-28 [1] purrr 0.3.4 2020-04-17 [3] R6 2.5.0 2020-10-28 [3] rappdirs 0.3.3 2021-01-31 [3] rcmdcheck 1.3.3 2019-05-07 [3] Rcpp 1.0.6 2021-01-15 [3] readr 1.4.0 2020-10-05 [3] rematch 1.0.1 2016-04-21 [3] remotes 2.2.0 2020-07-21 [3] rex 1.2.0 2020-04-21 [3] rhub * 1.1.1 2019-04-08 [1] rlang 0.4.10 2020-12-30 [3] rprojroot 2.0.2 2020-11-15 [1] rstudioapi 0.13 2020-11-12 [3] rvest 1.0.0 2021-03-09 [1] sessioninfo 1.1.1 2018-11-05 [3] stringi 1.5.3 2020-09-09 [1] stringr 1.4.0 2019-02-10 [3] targets 0.4.1 2021-04-22 [1] testthat * 3.0.1 2020-12-17 [3] tibble 3.1.1 2021-04-18 [1] tidyr 1.1.3 2021-03-03 [1] tidyselect 1.1.0 2020-05-11 [3] triebeard 0.3.0 2016-08-04 [3] urltools 1.7.3 2019-04-14 [3] usethis * 2.0.0 2020-12-10 [3] utf8 1.1.4 2018-05-24 [3] uuid 0.1-4 2020-02-26 [3] vcr * 0.6.0 2020-12-12 [3] vctrs 0.3.7 2021-03-29 [1] P weathercan * 0.5.0.9000 2021-03-03 [?] webmockr 0.7.4 2020-12-09 [3] whisker 0.4 2019-08-28 [3] whoami 1.3.0 2019-03-19 [1] withr 2.4.1 2021-01-26 [3] xml2 1.3.2 2020-04-23 [3] xmlparsedata 1.0.5 2021-03-06 [1] xopen 1.0.0 2018-09-17 [3] yaml 2.2.1 2020-02-01 [3] source CRAN (R 4.0.0) CRAN (R 4.0.2) CRAN (R 4.0.3) CRAN (R 4.0.3) CRAN (R 4.0.4) CRAN (R 4.0.0) CRAN (R 4.0.3) CRAN (R 4.0.4) CRAN (R 4.0.4) Github (ropensci/crul@a22b191) CRAN (R 4.0.0) CRAN (R 4.0.2) CRAN (R 4.0.4) CRAN (R 4.0.3) CRAN (R 4.0.0) CRAN (R 4.0.2) CRAN (R 4.0.3) CRAN (R 4.0.4) CRAN (R 4.0.0) CRAN (R 4.0.3) CRAN (R 4.0.3) CRAN (R 4.0.0) CRAN (R 4.0.2) CRAN (R 4.0.3) CRAN (R 4.0.2) CRAN (R 4.0.2) CRAN (R 4.0.3) CRAN (R 4.0.0) CRAN (R 4.0.2) CRAN (R 4.0.3) CRAN (R 4.0.3) CRAN (R 4.0.0) CRAN (R 4.0.4) CRAN (R 4.0.2) CRAN (R 4.0.4) CRAN (R 4.0.3) CRAN (R 4.0.3) CRAN (R 4.0.2) CRAN (R 4.0.4) CRAN (R 4.0.4) CRAN (R 4.0.3) CRAN (R 4.0.0) CRAN (R 4.0.2) CRAN (R 4.0.1) CRAN (R 4.0.0) CRAN (R 4.0.0) CRAN (R 4.0.3) CRAN (R 4.0.4) CRAN (R 4.0.0) CRAN (R 4.0.3) CRAN (R 4.0.3) CRAN (R 4.0.0) CRAN (R 4.0.3) CRAN (R 4.0.3) CRAN (R 4.0.0) CRAN (R 4.0.2) CRAN (R 4.0.0) CRAN (R 4.0.2) CRAN (R 4.0.3) CRAN (R 4.0.3) CRAN (R 4.0.3) CRAN (R 4.0.4) CRAN (R 4.0.0) CRAN (R 4.0.2) CRAN (R 4.0.0) CRAN (R 4.0.4) CRAN (R 4.0.3) CRAN (R 4.0.4) CRAN (R 4.0.4) CRAN (R 4.0.0) CRAN (R 4.0.2) CRAN (R 4.0.2) CRAN (R 4.0.3) CRAN (R 4.0.2) CRAN (R 4.0.0) CRAN (R 4.0.3) CRAN (R 4.0.4) Github (ropensci/weathercan@34f43d0) CRAN (R 4.0.3) CRAN (R 4.0.0) CRAN (R 4.0.2) CRAN (R 4.0.3) CRAN (R 4.0.0) CRAN (R 4.0.4) CRAN (R 4.0.0) CRAN (R 4.0.0) [1] /home/kevcaz/Applications/R_4_0 [2] /usr/local/lib/R/site-library [3] /usr/lib/R/site-library [4] /usr/lib/R/library P ── Loaded and on-disk path mismatch. ```
KevCaz commented 3 years ago

BTW, on my fork all jobs are currently failing https://github.com/KevCaz/weathercan/actions/runs/796508465.

KevCaz commented 3 years ago

Looks like this is simply due to the fact that the database is currently unavailable! :smile: https://climate.weather.gc.ca/error/dbdown_e.html Sorry, I should have checked that first! I'll wait till it gets back and then I'll close this!

KevCaz commented 3 years ago

This makes me think that it could useful to add a simple function to check if data are available, something like: 

  check_status <- function() {
    tmp <- httr::GET("https://climate.weather.gc.ca")
    chk <- tmp$url == "https://climate.weather.gc.ca/error/dbdown_e.html"
    if(chk) {
      message("service is down!")
    } else {
      message("data are available.")
    }
    invisible(!chk)
  }

just a thought.

steffilazerte commented 3 years ago

That's a beautiful idea! I wasn't aware of the dbdown site, which makes it nice and easy to check! Would you be interested in submitting a pull request (matching the normals branch, since that's the one I'm just about to merge in for an upcoming update).

I'd be very grateful!

KevCaz commented 3 years ago

Sure :smiley: . I'll think a bit about where this feature would be the most useful in the package, and then I'll open a PR, soon (tomorrow or this week-end!).

steffilazerte commented 3 years ago

Awesome! The packages is a bit messy, but in the normals branch (important to work on that one!), weather_html() (weather.R), normals_html() (normals.R), stations_dl_internal(), and normals_stn_list() (stations.R) are the four main access points for the ECCC. They all use the urls stored as options loaded by the .onLoad() in weathercan-pkg.R

Even if you just want to get it started, I'm happy to finish it up if you're busy. I appreciate anything you have time for :)