Week of 6/24/24

[mwdunlap2004]

Review of Thursday Meeting: Analysts showed table of precipitation datasets to show their comparisons. Also an example of a different way of looking at the storm sep code, to increase the number of storms it senses. Lots of talk about different packages, and the opportunity to explore and see if these examples have been done before.

• [x] Look at Hydro Packages, fasstr, EGRET, SF, raster, & IHA
• [x] Meet with DEQ modelers to introduce workflows in meta_model https://github.com/HARPgroup/meta_model/issues/59
  • [ ] Discuss and plan data model to store metrics/analysis plots
  • [x] Plan 1 chart/analytic per analyst and implement in meta_model workflow
• [ ] Read Literature about water flow, precipitation delays, and storm separation, and trying to find proven/studied ways to examine precipitation.
• [ ] Continue poking around Stormsep (ways to improve, other models to do it)
• [ ] Look into types of events where an increase in flow could be separate from precipitation events

[ilonah22]

Here are the tables and images from 6/20 summary:

[ilonah22]

Friday 6/21:

Today Michael is away so I read through a few papers about precipitation and streamflow delay and looked at the CRAN reference manual for the sf package.

[mwdunlap2004]

I actually did do some work today while we were driving, I mostly looked into some of the packages we had discussed namely fasstr and EGRET, I'm not sure how useful they are to us as they do things we've done before. Their way of looking at min and max flows though could be very interesting.

[durelles]

Look forward to hearing more specifics next week. Have a great weekend all, S

From: mwdunlap2004 @.> Date: Friday, June 21, 2024 at 4:30 PM To: HARPgroup/HARParchive @.> Cc: Scott, Durelle @.>, Assign @.> Subject: Re: [HARPgroup/HARParchive] Week of 6/24/24 (Issue #1288)

I actually did do some work today while we were driving, I mostly looked into some of the packages we had discussed namely fasstr and EGRET, I'm not sure how useful they are to us as they do things we've done before. Their way of looking at min and max flows though could be very interesting.

— Reply to this email directly, view it on GitHubhttps://github.com/HARPgroup/HARParchive/issues/1288#issuecomment-2183412636, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AC4K427ASADMB4DVPMYHVKDZISEM3AVCNFSM6AAAAABJUPKZMSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBTGQYTENRTGY. You are receiving this because you were assigned.Message ID: @.***>

[ilonah22]

From HARPgroup/model-meteorology/Workflow 1: simple 'lm()' analysis

• • usgs_data: Rscript to take in gage ID, (and time period?) and download flow data, save as csv
  • don't overwrite unless requested to
  • user calls script with desired filename (don't guess)
• bf_separation: Rscript to take a flow CSV as input, create the bf separaton plot, save it as a file
  • generate a BF plot for some time period. Maybe an arbitrary date? maybe the last year of the dataset?
  • User calls script with input file, and the name of the output baseflow graph(s)
• Research Rscript.exe and passing arguments and reading arguments inside the command prompt. See ex: https://github.com/HARPgroup/meta_model/blob/main/scripts/river/hsp_hydr_analysis.R
• (*bonus) lm_lastweek: Rscript to create lm stats

[mwdunlap2004]

This is an example of a function that takes in all these variables as well as a desired file path and outputs a csv, I've tried it with and without start dates, as well as changing the file path and it seems to work well.

usgs_data <- function(site_id, parameterCd, startDate = NULL, endDate = NULL, file.path){ if (is.null(c(startDate, endDate))) { spas <- dataRetrieval::readNWISdv(site_id, parameterCd) spas <- spas %>% rename(Q = X_00060_00003, Datetime = Date) } else{ spas <- dataRetrieval::readNWISdv(site_id, parameterCd, startDate, endDate) spas <- spas %>% rename(Q = X_00060_00003, Datetime = Date)

}

return(write.csv(spas, file.path)) }

usgs_data(site_id,parameterCd, startDate, endDate, file.path = "~/HarpData/HARParchive/HARP-2024-2025/test_data.csv")

[ilonah22]

Here is the baseflow separation function with inputs of csv file path, end file path, and desired filename. Output is baseflow separation plot in whichever form the user chooses:

bf_separation <- function(csv.path, end.path, filename){ water_data <- read.csv(csv.path) hdata = water_data %>% mutate(Qbase = gr_baseflow(Q, method = 'lynehollick', a = 0.925, passes = 3))

gp <- hdata |> ggplot() + geom_area(aes(as.Date(Datetime), Q), fill = 'steelblue', color = 'black') + geom_area(aes(as.Date(Datetime), Qbase), fill = 'orangered', color = 'black')

return(ggsave(filename= filename, plot = gp, path = end.path,limitsize = FALSE)) }

[durelles]

[like] Scott, Durelle reacted to your message:

---

From: Ilona Hughes @.> Sent: Tuesday, June 25, 2024 8:26:55 PM To: HARPgroup/HARParchive @.> Cc: Scott, Durelle @.>; Assign @.> Subject: Re: [HARPgroup/HARParchive] Week of 6/24/24 (Issue #1288)

Here is the baseflow separation function with inputs of csv file path, end file path, and desired filename. Output is baseflow separation plot in whichever form the user chooses:

bf_separation <- function(csv.path, end.path, filename){ water_data <- read.csv(csv.path) hdata = water_data %>% mutate(Qbase = gr_baseflow(Q, method = 'lynehollick', a = 0.925, passes = 3))

gp <- hdata |> ggplot() + geom_area(aes(as.Date(Datetime), Q), fill = 'steelblue', color = 'black') + geom_area(aes(as.Date(Datetime), Qbase), fill = 'orangered', color = 'black')

return(ggsave(filename= filename, plot = gp, path = end.path,limitsize = FALSE)) }

— Reply to this email directly, view it on GitHubhttps://github.com/HARPgroup/HARParchive/issues/1288#issuecomment-2189908511, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AC4K427CJYD5RBLPZ3WA3UTZJHHA7AVCNFSM6AAAAABJUPKZMSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBZHEYDQNJRGE. You are receiving this because you were assigned.Message ID: @.***>

[COBrogan]

@mwdunlap2004 I like the USGSdata function. It's simple and easy to use. One thing I noticed that I wanted to discuss was in that if statement. The if (is.null(c(startDate, endDate))){ ... } is a bit of a trap in R. is.null accepts the vector c(startDate,endDate). However, it does not apply is.null() to each element in the vector. Instead, it asks "Is this vector NULL?"_. Unless every element of the vector is NULL, the vector itself is defined. Therefore, is.null() will always return FALSE as long as either startDate or endDate is defined. In addition, you can see below that a vector defined via c() or other means automatically ignores NULLs and does not actually store them:

startDate <- "2020-02-18"
endDate <- NULL
is.null(c(startDate,endDate))
[1] FALSE

c(startDate,endDate)
[1] "2020-02-18"

Instead, you can structure this in any of the following manners or in some other apply case you may think of:

if(is.null(startDate) || is.null(endDate)){ ... }
if(any(is.null(startDate),is.null(endDate)){ ... }
if(unlist(lapply(list(startDate,endDate),FUN = is.null))){ ... }

Alternatively, remember that dataRetrieval::readNWISdv() has defaults for the startDate and endDate parameters already. According to ?readNWISdv, using an empty character string "" will automatically get the full data range (which you already know because you've used it here, but I'm just reiterating). So, instead of defaulting to NULLs in the arguments of usgs_data, we could instead default startDate and endDate to be the empty string "". Totally optional. Working with NULLs can be tricky, but rewarding. I found NULL variables to be really critical to getting interactive shiny applications to work and I seem to like using them in javaScript as well.

This is the fun part of creating functions. Everything works well when you test it out, but the moment someone else gets it, something breaks due to an unexpected input. It's all part of the process.