dataRetrieval: readNWIS

[lindsayplatt]

Parent: dataRetrieval: accessing timeseries data

• Available functions for pulling NWIS data
• Parameters available to specify location, time span, and type of data
• Exploring metadata
• Examples

[lindsayplatt]

date: "9999-11-01"

[lindsayplatt]

For readNWISuse:

Aggregate National Data POP,TW,PS,SD,L,I,TP,SI,M,C,A

Aggregate County and State Data

• AQ Aquaculture
• CO Commercial
• DO Domestic
• HY Hydroelectric Power
• IC Irrigation, Crop
• IG Irrigation, Golf Courses
• IN Industrial
• IT Total Irrigation
• LA Livestock (Animal Specialties)
• LI Livestock
• LS Livestock (Stock)
• MI Mining
• OI Other Industrial
• PC Thermoelectric Power (Closed-loop cooling)
• PF Fossil-fuel Thermoelectric Power
• PG Geothermal Thermoelectric Power
• PN Nuclear Thermoelectric Power
• PO Thermoelectric Power (Once-through cooling)
• PS Public Supply
• PT Total Power
• PT Total Thermoelectric Power
• RE Reservoir Evaporation
• TP Total Population
• WW Wastewater Treatment

Site Specific Water-Use Data

• WS Water Supply Public supply refers to the water delivered to several users for domestic, commercial, industrial, and thermoelectric power uses when the water is withdrawn by public and private water suppliers.
• DO Domestic Domestic water is that which is used for everyday household purposes, such as drinking, food preparation, bathing, washing clothes and dishes, flushing toilets, and watering lawns and gardens.
• CO Commercial Commercial water is that which is used by motels, hotels, restaurants, office buildings, other commercial facilities, and civilian and military institutions.
• IN Industrial Industrial water is used in facilities that manufacture products for purposes of processing, washing, and cooling.
• MI Mining Mining water is used in the preparations that are a part of mining activities such as the extraction of naturally occurring materials, dewatering, and milling.
• LV Livestock Livestock water use is water used for livestock watering, feed lots, dairy operations, and other on-farm needs. Livestock includes all animals other than fish.
• AQ Aquaculture Aquaculture water use is water used for fish hatcheries and fish farms.
• IR Irrigation Irrigation water use includes water used to irrigate public and private golf courses and all water artificially applied to farm and horticultural crops.
• TE Thermoelectric Thermoelectric power water use is water used in the process of generating thermoelectric power using fossil fuel, nuclear, or geothermal fuel sources.
• PH Hydroelectric Hydroelectric power generation refers to the water used in the generation of electricity at plants where the turbine generators are driven by falling water.
• ST Wastewater Treatment Wastewater treatment refers to the amount of water treated and released by facilities engaged primarily in the collection, treatment, and disposal of wastewater conveyed through a sewer system.
• RM Remediation Remediation water use is water that is either (1) pumped from a contaminated area to control the flow direction of contaminants in ground-water or (2) pumped, treated, and either used or returned to ground-water or surface-water.

[lindsayplatt]

@ldecicco-USGS we merged #146, did you get a chance to take a look on DEV? I'd like to close this issue if you've done your check on dev and approve.

[ldecicco-USGS]

Maybe make the waterYear example start on Oct. 1st so it's more obvious in the printout of head?

co_river_q_fall <- readNWISdv(siteNumber="09403850", parameterCd="00060", 
                              startDate="2014-10-01", endDate="2014-09-30")

[ldecicco-USGS]

I'd also change in the initial table from:

    Returns time-series data reported at fixed intervals (15-60 min).

to

    Returns time-series data reported from the USGS Instantaneous Values Web Service

They aren't guaranteed to be a fixed interval.

[ldecicco-USGS]

Maybe for parameterCd and statCd in the table, you could a links:

https://help.waterdata.usgs.gov/codes-and-parameters/parameters

https://help.waterdata.usgs.gov/code/stat_cd_nm_query?stat_nm_cd=%25&fmt=html

Also, just generally a link here would be a good addition: https://help.waterdata.usgs.gov/

[ldecicco-USGS]

I'm not a huge fan of Table 2...as it is I think it's more confusing than helpful. My first vote would be to take it out. The second vote would somehow indicate what functions the arguments are used in. Now I finally understand why you kept asking about type character can be "base", "corr", or "exsa"...by itself, that makes no sense. At least if you know that argument is specific to the rating function, you can read more in the rating's help file to make more sense of it. Same with statReportType...this table might make you think that would be an acceptable argument in the readNWISdv function (since readNWISdv aggregates by a stat type).

I also think the sentence: "Note that the web service name might differ from the dataRetrieval argument name, e.g. site is NWIS web service name but siteNumber is used in readNWIS functions." is not necessary. What we were trying to do with dataRetrieval is set a consistent user experience. They can use siteNumber all over (NWIS, WQP, NGWMN). If they're savvy enough to be checking the web service documentation and use site....that will work too. But by pointing that out here...I think that will just confuse people.

[ldecicco-USGS]

Also, seems to be some formatting problem : < name=“readnwisdata-county”

That's all I've got! Overall looks good!

[lindsayplatt]

1. I don't want the WY example to start in October because I wanted to show how the column was different between two water years. I will change the startDate to "2014-09-28" so you can see the change from 2014 to 2015 when you run head.
2. Changed readNWISuv definition
3. I cannot add a link to the arguments table for parameterCd or statCd because that table is auto-generated using formals and regexp to grab the descriptions. I added the links about parameterCd and statCd above Table 2, and the general help link is referenced at the beginning of the data discovery page.
4. I couldn't fit Table 1 and 2 together, but the idea is that each argument is defined in one location, and Table 1 tells you where it's used. I guess it could be confusing, but I kept wanting a summary like this as I was using dataRetrieval functions. I'm going to make a table like this for each of the function's individual sections. I think that would be more appropriate.
5. I guess I didn't realize that any webservice name could be used as the argument in your dataRetrieval function. I agree that this sentence could confuse people. It's gone!
6. Formatting issue -- I was missing an "a" in the HTML tag. It's now resolved.

Fixing the table issue, then a PR will be coming.

[ldecicco-USGS]

3. a. You'd have to do some coding, something like:

   
   Argument <- unique(allArguments)
   Description <- Description[!duplicated(allArguments)]

df <- data.frame(Argument, Description, stringsAsFactors = FALSE) df$Description[df$Argument == "parameterCd"] <- paste(df$Description[df$Argument == "parameterCd"], 'see: <a href"https://help.waterdata.usgs.gov/codes-and-parameters/parameters">NWIS help')

library(htmlTable) htmlTable(df, caption="Table 2. readNWIS argument definitions", rnames=FALSE, align=c("l","l"), col.rgroup = c("none", "#F7F7F7"), css.cell="padding-bottom: 0.5em; padding-right: 0.5em; padding-top: 0.5em;")

3. b. I really appreciate that it's neat to auto-generate Table 2....but my concerns still stand. As a stand-alone table, it's confusing, and possibly mis-leading that people will think all arguments are available in all functions. I don't understand why it wouldn't be more helpful to emphasize that people go to the help pages because there's a wealth of information there. 

3. c. If there's a compelling reason to keep Table 2....then split it up?:
You could modify the first table here:
https://github.com/USGS-R/dataRetrieval/wiki

Or, add a column with the appropriate functions:

Function <- c("readNWISdata", "readNWISdv", "readNWISgwl", "readNWISmeas", "readNWISpCode", "readNWISpeak", "readNWISqw", "readNWISrating", "readNWISsite", "readNWISstat", "readNWISuse", "readNWISuv") allArguments_list <- lapply(Function, function(f) { names(formals(f)) }) names(allArguments_list) <- Function

allArguments <- unlist(lapply(Function, function(f) { names(formals(f)) })) Description <- unlist(lapply(Function, function(fn){ unlist(lapply(names(formals(fn)), extract_arg_def, pkg="dataRetrieval", fn=fn)) }))

Argument <- unique(allArguments) Description <- Description[!duplicated(allArguments)] Functions <- c() for(i in Argument){ Functions <- c(Functions, paste(names(allArguments_list)[which(sapply(allArguments_list, function(x) i %in% x))], collapse = "\n")) }

df <- data.frame(Argument, Description, Functions, stringsAsFactors = FALSE)

library(htmlTable) htmlTable(df, caption="Table 2. readNWIS argument definitions", rnames=FALSE, align=c("l","l"), col.rgroup = c("none", "#F7F7F7"), css.cell="padding-bottom: 0.5em; padding-right: 0.5em; padding-top: 0.5em;")

[lindsayplatt]

@ldecicco-USGS my solution for your concern w/ Table 2 was to split it up. See #164.

[lindsayplatt]

I like the code to get links into the table! I'm going to add that.

[lindsayplatt]

I think this is good thanks to @ldecicco-usgs work in #167.