meta model components

[jdkleiner]

Model components

• scenario (e.g. hsp2_2022)
• segment (e.g. PM7_4820_0001)
• model (hspf_cbp6, hsp2_cbp6)
  • module (land, river, or all),
    • step (prep, run, link, analyze, or all),
      • plugin (00_init, 02_confluence, 03_upstream ... 99_tidy_up, or all)

river module (steps and plugins):

prep

• meta_model/models/hsp2_cbp6/river/prep/: Link

Component	Description	Link
00_init
01_run_rug
03_special_action
04_convert_hsp2
05_ann_load
06_ave_load
08_etm
09_runoff_dsn

run

• meta_model/models/hsp2_cbp6/river/run/: Link

Component	Description	Link
00_init	create tempdir if it doesn’t exist
02_confluence	handle confluence
03_upstream	make stream wdms: copy the eos wdm then add upstream to it
04_ps_sep_div
05_compare	compare calibration and scenario loads input to this river. modify comparison file
07_get_uci
08_run_river	This is what actually runs the river simulation (see notes below)
09_hydr_for_sed	Runs export_hsp_h5.R to export the hydr table as a csv and runs hsp_index_conversion.R to add date, year etc. columns to hydr csv	export_hsp_h5.R hsp_index_conversion.R
10_hydr_to_wdm	Runs csv_export_wdm_format.R to export the OVOL3 column from the hydr csv as a WDM-format csv	csv_export_wdm_format.R
12_res_typeII	Apply SPARROW Reservoir (Type II) Efficiency
13_scour_org
99_tidy_up

• 08_run_river Link
  • loads configuration file source $META_MODEL_ROOT/models/$MODEL/model_config Link
    • loads model scenario specifics source $MODEL_ROOT/config/control/script/${scenario}.con
      • cat this file: jdk@deq2:/opt/model/p6/vadeq/config/control/script$ cat hsp2_2022.con
      • sets HSP_EXEC variable: set HSP_EXEC = /usr/local/bin/hsp2 Note this can also point to hspf
  • sets h5 variable: set h5 = $seg'.h5'
  • executes $HSP_EXEC run $h5
    • Notes: This is equivalent to manually executing hsp2 run PL3_5250_0001.h5

link

• meta_model/models/hsp2_cbp6/river/link/ Link
  • Note the component numbering may change as we add components to this link directory

Component	Description	Link
01_export_ext	Runs export_hsp_h5.R 3 times to export demand and point source data (divr, diva, and ps) as individual csvs	export_hsp_h5.R
02_join_hydr	Uses join_col.R 3 times to join each of the above mentioned data columns to the hydr csv	join_col.R
03_convert_hydr	Runs hsp_hydr_conversion.R to process demand, withdrawal, point source etc., and exports hydr csvs as hourly data for the whole timeframe and daily data for water-year. Script also substitutes demand for withdrawal for hspf hydr csvs as a work-around to not having needed columns for wd	hsp_hydr_conversion.R
99_cleanup	Stashes the 2 hydr csv files XXXX_hydr.csv & XXXX_hydrd_wy.csv

analyze

• meta_model/models/hsp2_cbp6/river/analyze/: Link

Component	Description	Link
00_init	Creates the temp directory and export directories for hydr files and images/figures if they don't exist
04_hydr_metrics	Runs hsp_hydr_analysis.R to calculate metrics and export them to VAHydro	hsp_hydr_analysis.R
05_hydr_plots	Runs hsp_hydr_plots.R to generate figures and export them to VAHydro	hsp_hydr_plots.R
99_cleanup	Moves csvs to the export directory and hydr folder

How to run the river simulation:

• run_model Link
  • Iterates through each module, step, and plugin specified
  • How to use:
    • run_model [model] [scenario] [segment] [work_path] [modules] [run_steps] [script]
    • run_model hsp2_cbp6 hsp2_2022 PU0_6080_5620 auto river run 08_run_river
  • How to run the river simulation: https://github.com/HARPgroup/HARParchive/issues/736

Helpful tools:

• https://github.com/HARPgroup/cbp_wsm/issues/84
• get_config: jdk@deq2:/opt/model/p6/vadeq$ cat /usr/local/bin/get_config
  • get all contents from a config file: cbp get_config hsp2_2022 river

[jdkleiner]

Investigating Analytics Scripts

meta_model

• hspf:
  • summarize_river.R
    • Located here: deq2:/opt/model/p6/vadeq/run/export/summarize_river.R
    • Called by: meta_model/models/hspf_cbp6/river/analyze/02_hydr Link
    • Based on waterSupplyModelNode.R
      • Located here: om/R/summarize/waterSupplyModelNode.R
• hsp2:
  • summarize_river.R has been split into 2 separate scripts: Issue
  • summarize_values.R
    • Located here: HARParchive/HARP-2022-Summer/AutomatedScripts/summarize_values.R Link
  • summarize_figures.R
    • Located here: HARParchive/HARP-2022-Summer/AutomatedScripts/summarize_figures.R Link
  • hsp_hydr_analysis.R
    • Located here: HARParchive/HARP-2022-Summer/AutomatedScripts/hsp_hydr_analysis.R Link
    • Called by: meta_model/models/hsp2_cbp6/river/analyze/04_analysis Link
    • This is what's generating the current hsp2 output: example: http://deq1.bse.vt.edu/d.dh/om-model-info/6963682
• Q: Where do the 2 new scripts get called?
  • summarize_values.R & summarize_figures.R to be added to: meta_model/models/hsp2_cbp6/river/analyze/04_analysis?? Link
  • https://github.com/HARPgroup/HARParchive/issues/742
  • Ans:
    • See 04_hydr_metrcis - hsp_hydr_analysis.R
    • See 05_hydr_plots - hsp_hydr_plots.R
• Q: Do we also need to replace summarize_river.R in the hspf directory with these 2 new scripts? - Yes

[rburghol]

This is great stuff @jdkleiner @glenncampagna I forsee 04_analysis replaced by "04_hydr_metrics" and "05_hydr_plots" or something like that

[jdkleiner]

@rburghol @glenncampagna Thanks Rob! Thinking we should also do the same for the hspf_cbp6 model directory meta_model/models/hspf_cbp6/river/analyze/ adding "04_hydr_metrics" and "05_hydr_plots" for running the latest-greatest scripts?

ALSO one more thing we need to decide, I've been going through and manually testing the HARP scripts, looks like we diverged at some point between storing these as om_model_element vs om_water_model_node example below. Which do we want to use moving forward?

• Ans 2/15/23: om_water_model_node
  • See: https://github.com/HARPgroup/meta_model/blob/main/scripts/river/hsp_hydr_analysis.R
  • See PR: https://github.com/HARPgroup/HARParchive/pull/741/