search

DOI-USGS / dataRetrieval

This R package is designed to obtain USGS or EPA water quality sample data, streamflow data, and metadata directly from web services.
https://doi-usgs.github.io/dataRetrieval/
Other
259 stars 84 forks source link

Help understanding "failed to parse" warning from whatNWISdata #565

Closed jennymurphy closed 3 years ago

jennymurphy commented 3 years ago

Hi! I used the following line of code to retrieve sample information about a particular HUC:

wq1 <- whatNWISdata(huc = "08")

And I got this --> Warning messages: 1: 40 failed to parse. 2: 5 failed to parse.

I'm not really sure what these are telling me. Did something "go wrong" during the retrieval? Any insight on these messages? Thanks

ldecicco-USGS commented 3 years ago

Hmm....what I suspected was not actually the case... What is giving the warnings are converting the begin_date and end_date. There are dates in those column that are "1828" or "1983-05" and the code is expecting some combinations of years, months, and days. So right now you are getting NA's in those columns the dates don't have all 3 components (year, month, date).

In the "user-friendly-functions", there is a way to say convertType = FALSE (which would not try to convert the text to date). I'll think abut if there's a clear way to include that in the functions who use .... In the meantime, for this particular query, you could:

url <- "https://waterservices.usgs.gov/nwis/site/?seriesCatalogOutput=true&huc=08&format=rdb"
data_character <- importRDB1("https://waterservices.usgs.gov/nwis/site/?seriesCatalogOutput=true&huc=08&format=rdb", convertType = FALSE)

So.... bug/enhancement: Add convertType option to the functions whose arguments are ...

jennymurphy commented 3 years ago

That makes sense. Thanks so much.

ldecicco-USGS commented 3 years ago

I'm going to leave this open to remind myself of adding the convertType option