Water Availability Case Studies

glenncampagna commented 7 months ago

Script: https://github.com/HARPgroup/HARParchive/blob/master/HARP-2023-Summer/WaterAvailability/WaterAvailability_CaseStudies.Rmd

Description: A case study is a descriptive example of a particular area (ex. a watershed/region) where water availability is calculated. For that example watershed, we can take a detailed look at the individual calculations and resulting values generated.

From #818

Generalized Equation: $WA{CPL} = Q{LCPLdem} - (PoF * Q{LCPLbase}) + S{minCPL}/CPL$ ; Where:
- $CPL$ = critical period length, such as 30 days, 90 days, 7 days, etc.
- $WA_{CPL}$ = mean rate of available water remaining during critical period
- $Q_{LCPLdem}$ = L90 simulated for a given demand scenario
- $Q_{LCPLbase})$ = The L90 simulated in the zero demand scenario (with the same meterology).
- $S_{minCPL}$ = minimum storage simulated during the L90 period in the demand scenario.
- $PoF$ = % of flow that must remain in stream
- Note: $(PoF * Q{LCPLbase})$ could be rewritted as $Q{mif}$, where $Q_{mif}$ = minimum instream flow (if mif is NOT a PoF, and is knowable in a single period mean numeric form)
Error Analysis Form: $WA{CPL} = Q{LCPLdem} - (PoF * Q{LCPLbase}) + S{minCPL}/CPL$ - e ;
- where e = model error over critical flow period in mgd
- See also: #1092

Current Table Examples: https://github.com/HARPgroup/HARParchive/tree/master/HARP-2023-Summer/WaterAvailability/CaseStudiesTables

Table/Figure Descriptions (modified from original guidelines):

Table 1: Water Availability equation breakdown. This table answers: "Do all variables in the equation make sense?". Table contains Water Availability values and all variables/metrics impacting water availability for all segments with local or upstream storage. Illustrates available flow components: Qbaseline, Qmif, Qmodel and Qavail = Qmodel - Qmif ; where Qmif = 0.9 * Qbaseline. Storage has been routed downstream using fn_extract_basin for this table
Table 2: Case Studies. Shows segments that were specified as outlet segments in the script input/params. Storage has been routed downstream for this table
Table(s) 3: Case study watershed tables. A table of he available water calculated for each river segment in the watershed. The values of each variable in the equation would be the contents of the table, where rows would be each river segment and columns would be each variable. We would use this to check whether the equation inputs and outputs generally make sense.
Table(s) 4: Case study watershed tables, like the previous table, but separated for L30/L90 for better viewing alongside maps
Table 5: Smin Method Comparison. Compares the approximate and near-exact Smin values. Also quantifies the # of days outside the low-flow period the Storage minimum occurs if the 2 Smin values are different.
- Approximate method: lowest Storage value within the low-flow year
- Near-Exact method: lowest Storage value within the low-flow period (30 or 90 days) For the outside_pd columns:
- Positive: near-exact Smin happens AFTER the approximate Smin (Storage reaches its minimum before the low-flow period )
- Negative: near-exact Smin happens BEFORE the approximate Smin(Storage reaches its minimum after the low-flow period)
Table 6: Segments where Smin needs to be updated in VAhydro. Instances where the original Smin metrics exported to VAhydro are since outdated, indicated by the new approximate Smin values calculated in the RMD

Note: tables 5 and 6 are only created and exported when the input Smin_analysis is set to TRUE

Graph 1 : Graph of available flow compared to demand flow. Available flow represents water availability without accounting for storage. This graph is used this to consider whether our calculations make physical sense for the watershed.
Map 1 : Map showing WA for river segments. Ways to show this could be the fill transparency or color of the river segments. We'd want to highlight segments with negative water availability. It'd be useful to see this at multiple levels: region or state-wide as well as just the segments upstream of a certain basin or impoundment.

Notes/Discussion: particular notes about running this watershed calculation, any discussion about the results as noted above, would then get written down to accompany the tables/graphs and become the case study.

*Original guidelines contained in an email from Gouri on 2/1/2024

Impoundments for Case Studies:

Lake Manassas (PL0_5141_5140)
Lake Moomaw (JU3_6900_6950)
Lake Chesdin (JA5_7480_0001)
Lake Anna (YP2_6390_6330)
Carvin's Cove (OR3_7740_8271_carvins)
Jennings Randolph Lake (PU3_4451_4450)

Inputs/Parameters: outlet_list: list of riverseg IDs which are the Case Study watershed outlets demand_scenario: scenario ID for a demand scenario (ex. runid_11) baseline_scenario: scenario ID for a baseline scenario (ex. runid_0) mif_coef: Minimum Instream Flow (PoF) coefficient (ex. 0.9) Smin_analysis: indicates whether or not the user wants to see Smin methods compared (TRUE/FALSE) write_Qavailable: whether or not to write Qavailable metric to VAhydro (TRUE/FALSE) write_outputs: where to write/save tables (local, github)

Render Example:

rmarkdown::render("~/Desktop/GitHub/HARParchive/HARP-2023-Summer/WaterAvailability/WaterAvailability_CaseStudies.Rmd", 
                  output_file = "~/Desktop/HARPteam23/WAcasestudies_knit_runid11_runid0_0.9.docx",
                  output_format = "word_document",
                  params = list(
                      outlet_list = c("PL3_5360_5250", "JU4_7000_7300", "JA5_7480_0001", "YP3_6670_6720", "OR3_7740_8271", "PU4_3970_3890"),
                      demand_scenario = 'runid_11', 
                      baseline_scenario = 'runid_0', 
                      mif_coef = 0.9, 
                      Smin_analysis = TRUE,
                      write_outputs = c("local", "github")))

Tasks:

[ ] Add bulleted list on first page of knitted output document giving user important info: where to look for outputs, naming conventions, more input info (Smin switch), takeaways from tables
[ ] Investigate & fix case of days separating methods > 500 (Lake Philpott)
[x] Remove ability to write Qavailable -- no longer necessary
[ ] Can the outputs to the users local path be organized into folders the same way the GitHub outputs are?
[x] Short run-through of using this RMD during team meeting 7/24
[x] Save maps & plots to GitHub
[x] Send team summary for case study with links to review outputs quickly and easily
[x] Rename input to write_outputs to apply to tables, maps, and plots
[x] Locate original questions for WA and add them to this issue in table descriptions

Related Scripts and Issues:

Smin calculation & comparison: https://github.com/HARPgroup/HARParchive/blob/master/HARP-2023-Summer/Smin_methods.R
WA calculation: https://github.com/HARPgroup/HARParchive/blob/master/HARP-2023-Summer/WA_methods.R
Smin #1140
Water Availability #818

glenncampagna commented 6 months ago

Initial 3 Table Previews

glenncampagna commented 6 months ago

Additional Table/Map/Graph Ideas

Table with the riversegs with the lowest WA values in the state
- This would support the map I have planned showing WA by riverseg using fill colors

rburghol commented 6 months ago

Hey @glenncampagna these are really shaping up. In terms of showing the lowest WA in the state, I think that the overall map of WA should do that implicitly. Moreover, we would expect small streams to have low W.A., and large streams to have higher WA, so that comparison may not really be relevant.

Looking at table 3, etc., I think an important piece of context should be wd_cumulative_mgd. Including that will allow us to draw connections. Question: are you using 90% BASELINE as the target MIF for calculating the portion of WA that comes from flow? I suppose you should indicate the target MIF as well.

glenncampagna commented 6 months ago

@rburghol Ok sounds good, I'll retrieve and include the wd_cumulative_mgd values in tables. I understand what you're saying about the table being somewhat redundant to the map in terms of WA values; I guess my thinking of the table was that it could be useful to see other metrics for the segments with the lowest WA (baseflow, demand, withdrawal which we'll be adding) and if those make sense.

The in stream flow coefficient is being multiplied by the base flow metric in the Water Availability equation, yes. I currently have the coefficient as an input to the script which I forgot to include (set to 0.9 for now). Are you saying that it'd be helpful to include in table titles? I'd agree with this since the other 2 important inputs (demand and baseline scenarios) are included in the titles/headers

rburghol commented 6 months ago

Hey @glenncampagna thanks for your thoughts. I think a separate table of lowest avail would be redundant, though if we had streams with negative values that could be of interest for a separate table. Those streams would likely be ones without storage, so, you will have to make sure you're not filtering out only impoundment segments. What do you think about negative AW?

Thanks for confirming the 90% mif, and I was thinking add it as a column, i.e. L90 MIF, which would allow us to later reflect cases where we use ELFs to customize them by basin. This will make the table fairly busy so we may regret this :)

glenncampagna commented 6 months ago

@rburghol Gotcha. Are you wanting the column to contain the actual minimum in stream flow values (MIF coefficient * Baseflow metric) or just the coefficient we're using?

rburghol commented 6 months ago

Actual would be my first guess - what do you think pros/cons @gmahadwar @glenncampagna

gmahadwar commented 6 months ago

MIF: Would decide based on how often is there a MIF that's separate

Pro is that we see the actual number being multiplied in the case where the MIF is NOT a PoF. I'm referring to :
- $(PoF * Q{LCPLbase})$ could be rewritten as $Q{mif}$, where $Q_{mif}$ = minimum instream flow (if mif is NOT a PoF, and is knowable in a single period mean numeric form)
Con is that for the main cases where there is a PoF, we aren't showing the input variables of PoF and Q_{LCPLbase}
- In these cases it is helpful to see the PoF and Q_{LCPLbase}

Third table: Also agree on the wd_cumulative_mgd in the third table, I'm thinking about if there are other helpful columns like cumulative point source (I assume that's ps_cumulative_mgd) or the L30/L90 value itself for the rseg, and the rseg flow out

glenncampagna commented 5 months ago

Carvins question

When I use OR3_7740_8271_carvins to identify Carvins Cove as one of our case studies, fn_extract_basin fails to pull any upstream segments
Is this because this rseg ID is unique to the reservoir? We talked about wanting to see Spring Hollow upstream of Carvins @rburghol , but I do not with the methods used so far

rburghol commented 5 months ago

@glenncampagna - this is because there are no upstream segments from Carvins, it is a headwater, so where you want to do it is with the outlet being Roanoke River in Roanoke, which will then have Carvin's, Tinker Creek and all the Roanoke segments upstream of Carvins. Thus:

roaroa_data <- fn_extract_basin(wshed_data,'OR3_7740_8271')

glenncampagna commented 5 months ago

Carvins update, similar Lake Anna question

Thanks @rburghol , that worked well:
Ordering has been done so far by cumulative wd which doesn't give exactly the correct upstream -> downstream order we want. Next step will be ordering by rseg naming convention: [prefix]_[this_seg]_[downstream_seg]
It looks like a similar thing to Carvins happens for Lake Anna where no upstream segs are found:
Should I treat Lake Anna as a headwater also and use its downstream segment to pull upstream segments?

Update:

A quick test doesn't find segments upstream of Lake Anna when using the first segment downstream

lakeAnna_data <- fn_extract_basin(metric_data,'YP3_6330_6700')
lakeAnna_data$riverseg
[1] "YP2_6390_6330" "YP3_6330_6700"

glenncampagna commented 5 months ago

Graph preview for Lake Moomaw watershed (upstream & downstream) @gmahadwar

Qavailable = Qdemand - MIF*Qbaseline

gmahadwar commented 5 months ago

Thanks, @glenncampagna, can you write out the elements that go into Qavailable? Just adding to this comment is fine right now. I know you said it verbally but would be helpful for reference to explain this graph.

rburghol commented 5 months ago

Error ! object 'table_imp' not found testing with the following:

rmarkdown::render(paste0(github_location,"/HARP-2023-Summer/WaterAvailability/WaterAvailability_CaseStudies.Rmd"), 
                  output_file = paste0(export_path,"/WAcasestudies_knit_runid11_runid0_0.9.docx"),
                  output_format = "word_document",
                  params = list(
                      demand_scenario = 'runid_11', 
                      baseline_scenario = 'runid_0', 
                      mif_coef = 0.9))

glenncampagna commented 5 months ago

No baseline for Lake Anna watershed

The only case study we've chosen where it appears the baseline scenario has not been run yet is Lake Anna:

Should I check runid_100 also for baseline data? @rburghol I currently only check runid_0

glenncampagna commented 5 months ago

Lake Chesdin case study mystery basins

See #1236

glenncampagna commented 4 months ago

Where is the function vahydro_post_metric_to_scenprop() created? I don't believe we have ever loaded it in R before and only used it in R via the command line

Answer: Add source(paste(cbp6_location, "code/cbp6_functions.R", sep = "/")) to config.R

glenncampagna commented 4 months ago

Map/Table Preview

Lake Manassas:

[x] Table would be better if it was L30-specific for this map
[x] Indicate the column mapped in the flextable?

glenncampagna commented 4 months ago

Lake Chesdin WA L90 Map & Table + Questions

Are there any columns of the table that can be removed when displaying alongside this map?
It looks like there might be an incorrect 90-day base flow value for seg 7 in the table but the equation handles it correctly
Would it be helpful to display values in the table in the same units? (mg/mgd for all)

glenncampagna commented 4 months ago

Relating to the possible incorrect base flow value for a Lake Chesdin watershed segment mentioned in the comment above:

rburghol commented 4 months ago

Thanks for the heads up on this @glenncampagna -- I agree this looks suspicious. I am doing some model diagnostics now and then I'll rerun to see what gives.

rburghol commented 4 months ago

Glenn -- the bad segment was rerun and now has data that makes sense. I am rerunning the segments downstream of it now, and they should be done in a bout 30 minutes (2:30 EST). Thanks for seeing that anomaly!

rburghol commented 4 months ago

@glenncampagna Now, to your questions:

Can be removed? No, looking at them, they all seem relevant, and as our objective is to use the table to explain the equations, I think they all need to be there (but their names could be made human readable for this demonstration)
Segment 7 is now fixed, and the final watershed in the system is finishin the zero rerun. All segment in the Appomattox with impoundments have had their summary scripts rerun since PR https://github.com/HARPgroup/om/pull/545
MGD for all rate columns would be a really good idea (convert from CFS for flows), and then they will paior more nicely with MG for the Storage column.

glenncampagna commented 4 months ago

Lake Chesdin Updates

Thanks for getting back to me @rburghol , here's the updated visuals and table with the fixed segment

glenncampagna commented 4 months ago

Carvins Wshed Map & Table after Ordering by cumulative withdrawal

glenncampagna commented 2 months ago

Example with Outputs Linked

Render used:

rmarkdown::render("~/Desktop/GitHub/HARParchive/HARP-2023-Summer/WaterAvailability/WaterAvailability_CaseStudies.Rmd", 
                  output_file = "~/Desktop/HARPteam23/WAcasestudies_knit_runid11_runid0_0.9.docx",
                  output_format = "word_document",
                  params = list(
                      outlet_list = c("PL3_5360_5250", "JU4_7000_7300", "JA5_7480_0001", "YP3_6670_6720", "OR3_7740_8271", "PU4_3970_3890"),
                      demand_scenario = 'runid_11', 
                      baseline_scenario = 'runid_0', 
                      mif_coef = 0.9, 
                      Smin_analysis = TRUE,
                      write_Qavailable = FALSE, 
                      write_outputs = c("local", "github")))

Table 1: WA eqn breakdown

Table 2: Case study segments

Table 3 example: Case study watershed (Carvins)

Table 4 example: Case study watershed for L90 (Carvins) (matches map at the bottom)

Table 5: Smin method comparison

Table 6: Segments where Smin in VAhydro needs updating

Plot example (L90 for Carvins) https://github.com/HARPgroup/HARParchive/blob/master/HARP-2023-Summer/WaterAvailability/CaseStudiesPlots/WAcaseStudies_Plot_5_OR3_7740_8271_QvsQavail_L90.png

Map example (L90 for Carvins) https://github.com/HARPgroup/HARParchive/blob/master/HARP-2023-Summer/WaterAvailability/CaseStudiesMaps/WAcaseStudiesMapL90_5_OR3_7740_8271.png

HARPgroup / HARParchive