ropensci / rnoaa

R interface to many NOAA data APIs
https://docs.ropensci.org/rnoaa
Other
330 stars 84 forks source link

Error fetching `isd-history.csv` from HTTP #388

Open jrherr opened 3 years ago

jrherr commented 3 years ago
Session Info ```{r} ─ Session info ───────────────────────────────────────────────────────────────────────────────────── setting value version R version 4.0.4 (2021-02-15) os macOS Big Sur 10.16 system x86_64, darwin17.0 ui RStudio language (EN) collate en_US.UTF-8 ctype en_US.UTF-8 tz America/Chicago date 2021-04-26 ─ Packages ───────────────────────────────────────────────────────────────────────────────────────── ! package * version date lib source assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.0.0) cachem 1.0.4 2021-02-13 [1] CRAN (R 4.0.2) callr 3.7.0 2021-04-20 [1] CRAN (R 4.0.2) V cli 2.4.0 2021-04-26 [1] CRAN (R 4.0.4) colorspace 2.0-0 2020-11-11 [1] CRAN (R 4.0.2) crayon 1.4.1 2021-02-08 [1] CRAN (R 4.0.2) crul 1.1.0 2021-02-15 [1] CRAN (R 4.0.2) curl 4.3 2019-12-02 [1] CRAN (R 4.0.0) DBI 1.1.1 2021-01-15 [1] CRAN (R 4.0.2) desc 1.3.0 2021-03-05 [1] CRAN (R 4.0.2) devtools 2.4.0 2021-04-07 [1] CRAN (R 4.0.2) digest 0.6.27 2020-10-24 [1] CRAN (R 4.0.2) dplyr 1.0.5 2021-03-05 [1] CRAN (R 4.0.2) ellipsis 0.3.1 2020-05-15 [1] CRAN (R 4.0.0) fansi 0.4.2 2021-01-15 [1] CRAN (R 4.0.2) fastmap 1.1.0 2021-01-25 [1] CRAN (R 4.0.2) fs 1.5.0 2020-07-31 [1] CRAN (R 4.0.2) generics 0.1.0 2020-10-31 [1] CRAN (R 4.0.2) ggplot2 3.3.3 2020-12-30 [1] CRAN (R 4.0.2) glue 1.4.2 2020-08-27 [1] CRAN (R 4.0.2) gridExtra 2.3 2017-09-09 [1] CRAN (R 4.0.0) gtable 0.3.0 2019-03-25 [1] CRAN (R 4.0.0) hoardr 0.5.2 2018-12-02 [1] CRAN (R 4.0.2) httpcode 0.3.0 2020-04-10 [1] CRAN (R 4.0.2) jsonlite 1.7.2 2020-12-09 [1] CRAN (R 4.0.2) lifecycle 1.0.0 2021-02-15 [1] CRAN (R 4.0.3) lubridate 1.7.10 2021-02-26 [1] CRAN (R 4.0.2) magrittr 2.0.1 2020-11-17 [1] CRAN (R 4.0.2) memoise 2.0.0 2021-01-26 [1] CRAN (R 4.0.2) munsell 0.5.0 2018-06-12 [1] CRAN (R 4.0.0) pillar 1.6.0 2021-04-13 [1] CRAN (R 4.0.2) pkgbuild 1.2.0 2020-12-15 [1] CRAN (R 4.0.2) pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.0.0) pkgload 1.2.1 2021-04-06 [1] CRAN (R 4.0.2) prettyunits 1.1.1 2020-01-24 [1] CRAN (R 4.0.0) processx 3.5.1 2021-04-04 [1] CRAN (R 4.0.2) ps 1.6.0 2021-02-28 [1] CRAN (R 4.0.3) purrr 0.3.4 2020-04-17 [1] CRAN (R 4.0.0) R6 2.5.0 2020-10-28 [1] CRAN (R 4.0.2) rappdirs 0.3.3 2021-01-31 [1] CRAN (R 4.0.2) Rcpp 1.0.6 2021-01-15 [1] CRAN (R 4.0.2) remotes 2.3.0 2021-04-01 [1] CRAN (R 4.0.2) rlang 0.4.10 2020-12-30 [1] CRAN (R 4.0.2) rnoaa * 1.3.2.93 2021-04-26 [1] Github (ropensci/rnoaa@9b50009) rprojroot 2.0.2 2020-11-15 [1] CRAN (R 4.0.2) rstudioapi 0.13 2020-11-12 [1] CRAN (R 4.0.2) scales 1.1.1 2020-05-11 [1] CRAN (R 4.0.2) sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 4.0.0) testthat 3.0.2 2021-02-14 [1] CRAN (R 4.0.2) tibble 3.1.1 2021-04-18 [1] CRAN (R 4.0.2) tidyr 1.1.3 2021-03-03 [1] CRAN (R 4.0.3) tidyselect 1.1.0 2020-05-11 [1] CRAN (R 4.0.0) triebeard 0.3.0 2016-08-04 [1] CRAN (R 4.0.2) urltools 1.7.3 2019-04-14 [1] CRAN (R 4.0.2) usethis 2.0.1 2021-02-10 [1] CRAN (R 4.0.2) utf8 1.2.1 2021-03-12 [1] CRAN (R 4.0.4) vctrs 0.3.7 2021-03-29 [1] CRAN (R 4.0.2) withr 2.4.2 2021-04-18 [1] CRAN (R 4.0.2) XML 3.99-0.6 2021-03-16 [1] CRAN (R 4.0.4) xml2 1.3.2 2020-04-23 [1] CRAN (R 4.0.0) yaml 2.2.1 2020-02-01 [1] CRAN (R 4.0.0) [1] /Library/Frameworks/R.framework/Versions/4.0/Resources/library V ── Loaded and on-disk version mismatch. ```

I've been getting the following error for a while.

isd_stations(refresh = TRUE)

Error in file(file, "rt") : 
  cannot open the connection to 'https://www1.ncdc.noaa.gov/pub/data/noaa/isd-history.csv'
In addition: Warning message:
In file(file, "rt") :
  URL 'https://www1.ncdc.noaa.gov/pub/data/noaa/isd-history.csv': Timeout of 60 seconds was reached

I can grab the isd-history.csv file from the FTP site. I can't seem to find where to update this in the package. I'm not sure how this is called.

Any advice on how to update this?

Perhaps add this to issue #296?

Thanks for any help in advance!

sckott commented 3 years ago

Thanks for the issue @jrherr !

Will have a look

sckott commented 3 years ago

Looks like it was probably some temporary downtime for their servers. Wokring for me now. Does it still not work for you?

jrherr commented 3 years ago

Hi @sckott! Thanks for looking into this. The fetch "connects" now, but it's throwing this error:

rlang::last_error()
<error/tibble_error_assign_incompatible_size>
Assigned data `meteo_spherical_distance(...)` must be compatible with existing data.
x Existing data has 27996 rows.
x Assigned data has 0 rows.
x Only vectors of size 1 are recycled.

I'm looking into it... Thanks again!

sckott commented 3 years ago

Assigned data meteo_spherical_distance(...) must be compatible with existing data.

hmm, not sure about this. I don't see how that stack trace would come from a call to isd_stations(). is that from a different function call that failed?