Closed lindsayplatt closed 2 years ago
Ugg...yeah, that's new. It also doesn't work when the user asks for a pcode that's not in the parameterCdFile. Usually, that would kick off a call to the service. Now you get a similar html result.
I believe the API (nwis.waterdata.usgs.gov) used for retrieving the parm_cd info has changed. The error comes out of the importRDB1 function (called when you use the 'all' option for the parm_cd). The importRDB1 function gets called elsewhere when the parm_cd doesnt exist in the cached/packaged file dataRetrieval is using (parameterCdFile) and I've verified the error persists there as well.
The data you are seeing is the html from the help page because the current fullURL that's used to retrieve all the parm_cd info gets forwarded there. That lead me to believe the fullURL isn't going to work anymore, likely because they've changed the way the API works. I ended up replacing the line using the importRDB1 in the readNWISpCode function with the following: parameterData <- importRDB1("https://help.waterdata.usgs.gov/code/parameter_cd_query?fmt=rdb&group_cd=%", asDateTime = FALSE)
This is a patch-work solution for myself for only the "all" parm_cd option, just as an FYI. The column names will comeback differently than the previous way of retrieving the parm_cd info, so that will have to be accounted for.
Hey Laura, I had some free time to get things functioning again. If you change the pkg.env$pCode to "https://help.waterdata.usgs.gov/code" in the setAccess.R file and replace the readNWISpCode function with the contents of the text file I've attached, everything should be back up and running. pcode_fix.txt
Just a heads up, I checked the setAccess.R file and it has the "/parameter_cd_query" in it. That base path only works for the "all" selection. For the individual pulls it'd need to be "/parameter_cd_nm_query". Thats why I left the base path as "https://help.waterdata.usgs.gov/code" and then appended them individually for those use cases within the readNWISpCode function.
Ah OK, I'll futz more then, the drURL
code messes up the spot of the question mark, so I kept getting a 500 with the "https://help.waterdata.usgs.gov/code"
I think the latest pull request took care of the single-pcode issue. Please install, play around with it, and let me know if anything looks amiss.
remotes::install_github("USGS-R/dataRetrieval")
paramINFO <- readNWISpCode(c('00060','01075','00931', NA))
param_all <- readNWISpCode("all")
This seems to be working now! param_all
has 24,200 rows 😮
Describe the bug
readNWISpCode("all")
is no longer working. No longer returns a nicely parsed dataset.To Reproduce I am using version 2.7.10 (but colleagues experienced this on 2.7.1). When you run
readNWISpCode("all")
, you do not get a nicely parsed table, but instead the start of an HTML webpage:Expected behavior I expect to see a table akin to what is returned from
parameterCdFile
. I instructed colleagues to useparameterCdFile
instead for now to the get all the pcode info into a table.Session Info Using
dataRetrieval
version 2.7.10.Additional context Colleagues first told me about this failing yesterday, but I don't know when it worked last to say that yesterday was the first day. Could it be related to the Drupal 9 switch?