ropensci / rnoaa

R interface to many NOAA data APIs
https://docs.ropensci.org/rnoaa
Other
328 stars 84 forks source link

token not working #370

Closed joshualerickson closed 3 years ago

joshualerickson commented 4 years ago
Session Info ```r devtools::session_info() - Session info ------------------------------------------------------------------------------------------------------- setting value version R version 3.5.3 (2019-03-11) os Windows >= 8 x64 system x86_64, mingw32 ui RStudio language (EN) collate English_United States.1252 ctype English_United States.1252 tz America/Denver date 2020-09-02 - Packages ----------------------------------------------------------------------------------------------------------- package * version date lib source assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.5.3) backports 1.1.5 2019-10-02 [1] CRAN (R 3.5.3) callr 3.4.3 2020-03-28 [1] CRAN (R 3.5.3) cli 2.0.2 2020-02-28 [1] CRAN (R 3.5.3) colorspace 1.4-1 2019-03-18 [1] CRAN (R 3.5.3) crayon 1.3.4 2017-09-16 [1] CRAN (R 3.5.3) crul 1.0.0 2020-07-30 [1] CRAN (R 3.5.3) curl 4.3 2019-12-02 [1] CRAN (R 3.5.3) desc 1.2.0 2018-05-01 [1] CRAN (R 3.5.3) devtools 2.3.1 2020-07-21 [1] CRAN (R 3.5.3) digest 0.6.25 2020-02-23 [1] CRAN (R 3.5.3) dplyr 1.0.0 2020-05-29 [1] CRAN (R 3.5.3) ellipsis 0.3.0 2019-09-20 [1] CRAN (R 3.5.3) fansi 0.4.1 2020-01-08 [1] CRAN (R 3.5.3) fs 1.3.2 2020-03-05 [1] CRAN (R 3.5.3) generics 0.0.2 2018-11-29 [1] CRAN (R 3.5.3) ggplot2 3.3.2 2020-06-19 [1] CRAN (R 3.5.3) glue 1.3.2 2020-03-12 [1] CRAN (R 3.5.3) gridExtra 2.3 2017-09-09 [1] CRAN (R 3.5.3) gtable 0.3.0 2019-03-25 [1] CRAN (R 3.5.3) hoardr 0.5.2 2018-12-02 [1] CRAN (R 3.5.3) httpcode 0.3.0 2020-04-10 [1] CRAN (R 3.5.3) jsonlite 1.6.1 2020-02-02 [1] CRAN (R 3.5.3) lifecycle 0.2.0 2020-03-06 [1] CRAN (R 3.5.3) lubridate 1.7.4 2018-04-11 [1] CRAN (R 3.5.3) magrittr 1.5 2014-11-22 [1] CRAN (R 3.5.3) memoise 1.1.0 2017-04-21 [1] CRAN (R 3.5.3) munsell 0.5.0 2018-06-12 [1] CRAN (R 3.5.3) pillar 1.4.6 2020-07-10 [1] CRAN (R 3.5.3) pkgbuild 1.1.0 2020-07-13 [1] CRAN (R 3.5.3) pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 3.5.3) pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.5.3) prettyunits 1.1.1 2020-01-24 [1] CRAN (R 3.5.3) processx 3.4.2 2020-02-09 [1] CRAN (R 3.5.3) ps 1.3.2 2020-02-13 [1] CRAN (R 3.5.3) purrr 0.3.3 2019-10-18 [1] CRAN (R 3.5.3) R6 2.4.1 2019-11-12 [1] CRAN (R 3.5.3) rappdirs 0.3.1 2016-03-28 [1] CRAN (R 3.5.3) Rcpp 1.0.4.6 2020-04-09 [1] CRAN (R 3.5.3) remotes 2.2.0 2020-07-21 [1] CRAN (R 3.5.3) RevoUtils * 11.0.3 2019-05-10 [2] local RevoUtilsMath * 11.0.0 2019-05-08 [2] local rlang 0.4.6 2020-05-02 [1] CRAN (R 3.5.3) rnoaa * 1.1.0 2020-07-08 [1] CRAN (R 3.5.3) rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.5.3) rstudioapi 0.11 2020-02-07 [1] CRAN (R 3.5.3) scales 1.1.1 2020-05-11 [1] CRAN (R 3.5.3) sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.5.3) stringi 1.4.6 2020-02-17 [1] CRAN (R 3.5.3) stringr 1.4.0 2019-02-10 [1] CRAN (R 3.5.3) testthat 2.3.2 2020-03-02 [1] CRAN (R 3.5.3) tibble 3.0.1 2020-04-20 [1] CRAN (R 3.5.3) tidyr 1.1.0 2020-05-20 [1] CRAN (R 3.5.3) tidyselect 1.1.0 2020-05-11 [1] CRAN (R 3.5.3) triebeard 0.3.0 2016-08-04 [1] CRAN (R 3.5.3) urltools 1.7.3 2019-04-14 [1] CRAN (R 3.5.3) usethis 1.6.1 2020-04-29 [1] CRAN (R 3.5.3) vctrs 0.3.1 2020-06-05 [1] CRAN (R 3.5.3) withr 2.2.0 2020-04-20 [1] CRAN (R 3.5.3) XML 3.99-0.3 2020-01-20 [1] CRAN (R 3.5.3) xml2 1.3.2 2020-04-23 [1] CRAN (R 3.5.3) yaml 2.2.1 2020-02-01 [1] CRAN (R 3.5.3) [1] C:/Users/joshualerickson/Documents/R/win-library/3.5 [2] C:/Program Files/Microsoft/R Open/R-3.5.3/library ```

I'm having trouble connecting to the Web Services API via the token I received. I work for the USDA so I don't know if it has to do with a network issue or if there was something wrong on my end? I followed the introduction vignette but kept getting the error below. Thank you. I hope I can get this to work because the package looks awesome!

library(rnoaa)

options(noaakey = "key-emailed")

ncdc(datasetid = 'PRECIP_HLY', locationid = 'ZIP:28801', datatypeid = 'HPCP', limit = 5, token =  "key-emailed")
#> Error in curl::curl_fetch_memory(x$url$url, handle = x$url$handle): schannel: next InitializeSecurityContext failed: SEC_E_ILLEGAL_MESSAGE (0x80090326) - This error usually occurs when a fatal SSL/TLS alert is received (e.g. handshake failed).

reprex::reprex()
#> No input provided and clipboard is not available.
#> Rendering reprex...
sckott commented 4 years ago

thanks for. your report @joshualerickson having a look

sckott commented 4 years ago

it's likely a windows specific issue with the curl library used. what's the output of curl::curl_version() ?

joshualerickson commented 4 years ago

Hey thanks for the quick response, below is the curl_version info.

curl::curl_version()
#> $version
#> [1] "7.64.1"
#> 
#> $ssl_version
#> [1] "(OpenSSL/1.1.1a) Schannel"
#> 
#> $libz_version
#> [1] "1.2.8"
#> 
#> $libssh_version
#> [1] "libssh2/1.8.2"
#> 
#> $libidn_version
#> [1] NA
#> 
#> $host
#> [1] "x86_64-w64-mingw32"
#> 
#> $protocols
#>  [1] "dict"   "file"   "ftp"    "ftps"   "gopher" "http"   "https"  "imap"  
#>  [9] "imaps"  "ldap"   "ldaps"  "pop3"   "pop3s"  "rtsp"   "scp"    "sftp"  
#> [17] "smtp"   "smtps"  "telnet" "tftp"  
#> 
#> $ipv6
#> [1] TRUE
#> 
#> $http2
#> [1] FALSE
#> 
#> $idn
#> [1] TRUE
joshualerickson commented 4 years ago

Followed this link https://github.com/ropensci/rgbif/issues/272# step-by-step with no success. I'm not sure what's going on with my end but want to figure it out for sure.

sckott commented 4 years ago

which things did you try in that issue? all of them?

joshualerickson commented 4 years ago

yeah i tried all of them, even uninstalled rnoaa and installed_github(). Thought it might have worked with installing by 'source' but that never got rid of that error... Not sure what's going on.

sckott commented 4 years ago

install what by source?

joshualerickson commented 4 years ago

curl with Rtools

joshualerickson commented 4 years ago

If I check with other sites (e.g. USGS, SNOTEL) the SSL/TLS handshake is complete. See below, but when I run the noaa.gov site it is not recognized. Going to contact IT on monday and see if there is something that we can do on our end to change this error. Thanks.

image image

joshualerickson commented 4 years ago

I think I'm getting closer to the problem... Need to change internet properties to allow for TLS 1.3; however, needs administrative priveledges to do so (wait until monday). Here is a picture of what happens when I search https://www.ncdc.noaa.gov/cdo-web/webservices/v2 in windows explorer (works perfect in chrome). Hopefully this will fix the error.

TLS_WI
sckott commented 3 years ago

Have you tried from different locations? That is, are you in an office at work? That can cause this kind of problem.

sckott commented 3 years ago

just tried the example code above on my windows VM running Windows 10 with R 4.0.2 and the same version of rnoaa, but with the required parameters startdate/enddate and it worked fine.

the only difference on my Windows VM in curl I could see was I have a different libz version:

curl::curl_version()
#> $libz_version
#> [1] "1.2.11"

I don't know if that should make a difference or not

You are also running a pretty old version of R, from March 2019. I don't think that would help, but it also doesn't hurt to have the latest R unless you need to stick to that R version for reproducibility reasons

joshualerickson commented 3 years ago

Hey Scott, Thanks for the feedback. I installed a new version of R yesterday (4.0.2) and updated curl (see below). I've tried it on my internet at home but with no luck. I talked with IT today and apparently it's getting elevated to 'security', so will see how long that takes... In the meantime the IT guy said this below,

"So it was on August 5 desktop was notified of the SSL and TLS error. That there was noting we could do other than to tell out users to call the website vendors. I did see your ticket and updated with that information and sent it to security. Its possible they wont do a dam thing and dump it back on us. IF that happens, please let me know."

Thanks Scott again for helping me through this!

curl::curl_version()$libz_version
#> [1] "1.2.11"
sckott commented 3 years ago

Sorry for delay. Any news on this?

Another option is that the data you seek may be available through other NOAA sources. Seems you're after hourly precip data. Do any of these other data sources have the same data maybe https://github.com/ropensci/rnoaa#data-sources-in-rnoaa ?

joshualerickson commented 3 years ago

Hey, yeah they added the TLS 1.0 abilities to my computer but with no success. I'm still getting the same error in the initial issue I posted. I've reinstalled rcurl and rnoaa with a new version of Rstudio and tried the TLS workaround. I tried other functions in rnoaa like ghcnd() and still get the same error. Honestly, I've over my head on this stuff, if you couldn't tell... So, I think I might install R on my wifes computer and then diagnose from there, e.g. compare settings etc.

The reason I want to get it to work: I'm working on a shiny app that takes in USGS and Snotel stations (hopefully weather stations as well) and then provides the user with a variety of graphs and tables from which they can explore and assist with real-time decision making in the usda-forest service. Thus, I was wanting to get the weather stations added as well so we could bring together even more hydroclimatic data. Here's the link to the ongoing beta version just to give you an idea of what I was wanting to do with the weather station data https://joshualerickson.shinyapps.io/app_build/. So, to answer your "hourly precip data" question in long way... I think daily values would be the best for this app; however, I would like to explore the hourly at some point.

Thanks Scott!

sckott commented 3 years ago

Sadly, I've got no more ideas. If I were you tried some of the other functions in rnoaa that do not get data from the NCDC API, because perhaps the problem is with the NCDC API server.