New Watershed Template - Githubissues

rburghol commented 1 year ago

Tasks

[ ] Set up subwatershed river segment template (may be universal?)
[ ] Set up main watershed template (if different from subwatershed, probably WILL be)
[ ] Set up facility template to test WD + PS integration https://github.com/HARPgroup/vahydro/issues/844

Prototype Development

Call this to set up easier access to Rscript.exe when running on windows:

PATH="$PATH:/c/usr/local/bin/R-4.2.2/bin"
export PATH
vahydro_path="/c/usr/local/home/git/vahydro/R/modeling/json"

Templates:
- River Segment (was JL1_6560_6440_beaver_creek, now JL1_6562_6560)
  - Beaver Creek, Crozet
    - vahydro-1.0: om-model-info/4431664
    - cbp-6.1: om-model-info/7047664
    - comid='8567219' # really the outlet is this: 8567221
    - intake lat/lon: 38.0713888889, -78.6525
  - How to create a json nested watershed for use in hsp2 ops model:
    - Create json for Beaver Creek:
    - BC just upstream of the lake: Rscript.exe ${vahydro_path}/nhdplus_nested.R 8567219 JL1_6562_6560
    - BC at the Mechums Confluence: Rscript.exe ${vahydro_path}/nhdplus_nested.R 8566805 JL1_6562_6560a
    - Create json for Mechums River at Rivanna Confluence: Rscript.exe ${vahydro_path}/nhdplus_nested.R 8566737 JL1_6560_6440
    - Create json for Mechums River at Rivanna Confluence but do NOT include Beaver creek as that is a subwatershed: Rscript.exe ${vahydro_path}/nhdplus_nested.R 8566737 JL1_6560_6440 8566805
See the local channel code development here; #16
- Facility:
  - vahydro-2.0 template: om-model-info/7118952
  - Crozet WTP
    - vahydro-1.0: om-model-info/4829113
    - cbp-6.1: om-model-info/7118959
    - Currently, just running the Crozet 1.0 version with the new prototype stream

Copy relevant pieces from old model

# the channel component:
drush scr modules/om/src/om_copy_subcomp.php cmd dh_properties 4431664 dh_properties 7047664 "local_channel"

# the impoundment component:
drush scr modules/om/src/om_copy_subcomp.php cmd dh_properties 4431664 dh_properties 7047664 "impoundment"

# Child listeners
drush scr modules/om/src/om_copy_subcomp.php cmd dh_properties 4431664 dh_properties 7047664 "Listen to Children"

Test Run

#river_pid = 4431664
river_pid = 7047664 # new prototype
# el_pid = 4829113
el_pid = 7118959 # new prototype

# run code from https://github.com/HARPgroup/HSPsquared/blob/devstateom/tests/testcbp/HSP2results/test_object_json.py

odb = model_object_cache['/STATE/RCHRES_R001/Listen on Parent']
ddb = model_data['RCHRES_R001']['Listen on Parent']
ddb.keys()
bc = ddb.get('broadcast_channel')

river = model_object_cache['/STATE/RCHRES_R001']
river.inputs
lc = model_data['RCHRES_R001']['local_channel']
lco = model_object_cache["/STATE/RCHRES_R001/local_channel"]
cmp = model_object_cache["/STATE/RCHRES_R001/current_monthly_pct"]
lc_Qout = lco.w_var_values['Qout']

# facility data
fdat = model_data['RCHRES_R001']['Crozet Beaver Creek']
fdat = model_data['RCHRES_R001']['Crozet Beaver Creek']

# facility object
fac = model_object_cache["/STATE/RCHRES_R001/Crozet Beaver Creek"]
fac.get_state('flowby')
base_demand = model_object_cache[fac.find_var_path("base_demand_mgd")]
wd_mgy = model_object_cache[base_demand.find_var_path('wd_mgy')]
fac.get_state('base_demand_mgd')

wd_mgy = model_object_cache[river.find_var_path('local_channel')]

Qr = model_object_cache[river.inputs['Qreach']]

# do we have the standard HSP2 stuff?
river.find_var_path("IVOLin")
# False - no!

rburghol commented 1 year ago

ftable_path="$CBP_ROOT/input/param/river/${PARAMS}/ftables/"
Rscript run/resegment/ftable_creation.R JL1_6560_6440_beaver_creek local_channel $ftable_path
cp $ftable_path/JL1_6560_6440_beaver_creek.ftable $ftable_path/${subshed}.ftable

cbp run_rug.csh subsheds $subshed

rburghol commented 1 year ago

Black Creek Reservoir

fgrep JL1_7200_7250_black_creek_rsvr /opt/model/HARParchive/HARP-2022-Summer/AutomatedScripts/SubshedsCreation/subshed_list.csv 
# 5758414,JL1_7200_7250_black_creek_rsvr,3.22
# run the resegment script
./run/resegment/add_sub_watershed.bat JL1_7200_7250_black_creek_rsvr JL1_7200_7250 cbp-6.0 subsheds 3.22
subshed="JL1_7202_7200" # from output 

ftable_path="$CBP_ROOT/input/param/river/${PARAMS}/ftables/"
Rscript run/resegment/ftable_creation.R JL1_7200_7250_black_creek_rsvr local_channel $ftable_path
cp $ftable_path/JL1_7200_7250_black_creek_rsvr.ftable $ftable_path/${subshed}.ftable

cbp run_rug.csh subsheds JL1_7202_7200

ls tmp/uci/river/subsheds/${subshed}.uci

# now run it all 
/opt/model/meta_model/run_model hsp2_cbp6 subsheds $subshed auto river prep
/opt/model/meta_model/run_model hsp2_cbp6 subsheds $subshed auto river run

Errors. Debug:

Error: opening wdm= -2 ../../../tmp/wdm/land/fnp/vadeq_2021/fnp_H51003_JL1_6562_6560.wd

# remove error in last temp dir
rm $CBP_ROOT/tmp/scratch/4062/problem

# prep steps
# 00_init  01_run_rug  03_special_action  04_convert_hsp2  05_ann_load  06_ave_load  08_etm  09_runoff_dsn

/opt/model/meta_model/run_model hsp2_cbp6 subsheds $subshed $CBP_ROOT/tmp/scratch/4062 river prep 03_special_action
/opt/model/meta_model/run_model hsp2_cbp6 subsheds $subshed $CBP_ROOT/tmp/scratch/4062 river prep 04_convert_hsp2
/opt/model/meta_model/run_model hsp2_cbp6 subsheds $subshed $CBP_ROOT/tmp/scratch/4062 river prep 05_ann_load  
# error in ann_load step 
# Could not find segment JL1_6562_6560 in file ../../../config/catalog/geo/p600/land_water_area.csv

Ahh! Geo for land scenario must match (for ann_load to run that is)

Run New land (which includes the full GEO)

lsegs=`cbp get_landsegs JL1_6562_6560 `
# now use new sbatch technique to run land simultaneously
for i in $lsegs; do
  echo "sbatch /opt/model/meta_model/run_model hspf_cbp6 vahydro_2022 $i auto land "
  sbatch /opt/model/meta_model/run_model hspf_cbp6 vahydro_2022 $i auto land 
done

rburghol commented 1 year ago

Beaver Creek

Run New land (which includes the full GEO)

Run Land

MODEL_ROOT=/opt/model/p6/vadeq
META_MODEL_ROOT=/opt/model/meta_model
IGNORE_PROBLEMS=0 # set this to 1 if you want to plow forward regardless of errors, not advisable but sometimes...
export MODEL_ROOT META_MODEL_ROOT IGNORE_PROBLEMS

lsegs=`cbp get_landsegs JL1_6562_6560 `
# now use new sbatch technique to run land simultaneously
for i in $lsegs; do
  echo "sbatch /opt/model/meta_model/run_model hspf_cbp6 vahydro_2022 $i auto land "
  sbatch /opt/model/meta_model/run_model hspf_cbp6 vahydro_2022 $i auto land 
done

Run River

/opt/model/meta_model/run_model hsp2_cbp6 subsheds JL1_6562_6560 auto river

rburghol commented 1 year ago

Did not find the model. Debug:

river_segment_name = "JL1_6562_6560"
scenario_name = "subsheds"
input_file_path="/media/model/p6/out/river/subsheds/hydr/"
image_directory_path="/media/model/p6/out/river/subsheds/hydr/images/"
model_version="cbp-6.0"

rburghol commented 1 year ago

Need to find model better -- but this is not a good way -- so, we update the propcode for the corresponding segment feature to match the new segment id, then everything works.

but just for posterity, here it is:

model_riverseg <- RomProperty$new(
  ds,
  list(
    propname='riverseg',
    entity_type="dh_properties", 
    propcode=rseg_name
  ), 
  TRUE
)

moriverseg <- ds$get_prop(
  list(
    propname='riverseg',
    entity_type="dh_properties", 
    propcode='PS2_5560_5100'
  )
)
candidates = as.array(moriverseg$featureid)

for (i in candidates) {
   modelq = list(pid=i, varkey = 'om_water_model_node', propcode = model_version )
   model = RomProperty$new(ds,modelq, TRUE)
  if (!is.boolean(model)) {
    if (!is.na(model$pid)) {
     if (model$propcode == model_version ) {
        break;
     }
    }
  }
}

rburghol commented 6 months ago

Run

BC:

cd /opt/model/p6/vadeq/
i=JL1_6562_6560
. hspf_config
scenario=subsheds
/opt/model/meta_model/run_model hsp2_cbp6 $scenario $i auto river
# Ex: run just a single su-step
# /opt/model/meta_model/run_model hsp2_cbp6 $scenario $i tmp/scratch/rwb1_24 river 02_confluence

Mechums

cd /opt/model/p6/vadeq/
i=JL1_6560_6440
. hspf_config
scenario=subsheds
# use /opt/model/p6/vadeq/tmp/scratch/rwb instead of auto to preserve the data after running
/opt/model/meta_model/run_model hsp2_cbp6 $scenario $i auto river

Run summary

Beaver Creek

rmarkdown::render('/usr/local/home/git/vahydro/R/OWS_summaries/ws_model_summary.Rmd',
                  output_file = '/WorkSpace/modeling/projects/james_river/rivanna/beaver_hsp2/cia_vahydro_subsheds.docx',
                  params = list( doc_title = ("Test HSP2 Model Summary"),
                                 rseg.file.path = c("http://deq1.bse.vt.edu:81/data/proj3/out/runlog600.351963.log", "http://deq1.bse.vt.edu:81/p6/out/river/subsheds/hydr/JL1_6562_6560_hydrd_wy.csv" ),
                                 rseg.hydrocode = c("vahydrosw_wshed_JL1_6562_6560","vahydrosw_wshed_JL1_6562_6560"),
                                 rseg.ftype = c("vahydro","vahydro"),
                                 rseg.model.version = c("vahydro-1.0","cbp-6.1"),
                                 runid.list = c("runid_600","subsheds"),
                                 rseg.metric.list = c("7q10", "l90_Qout", "l30_year")
                  )
)

rburghol commented 6 months ago

Difficult Run

Rscript.exe ${vahydro_path}/nhd_channel_model_props.R 38.97503 -77.23546
# yields outlet 4509202
Rscript.exe ${vahydro_path}/nhdplus_nested.R 4509202 PM7_4581_4580

rburghol commented 5 months ago

cd /opt/model/p6/vadeq/
i=PM7_4620_4580
. hspf_config
scenario=subsheds
# use /opt/model/p6/vadeq/tmp/scratch/rwb instead of auto to preserve the data after running
/opt/model/meta_model/run_model hsp2_cbp6 $scenario $i auto river

make facility json Rscript.exe ${vahydro_path}/riverseg_facilities.R PM7_4620_4580 11271.7 4824325 vahydro-1.0 PM7_4620_4580.json

HARPgroup / HSPsquared

New Watershed Template #58

Tasks

Prototype Development

Copy relevant pieces from old model

Test Run

Run Land

Run River

Run

BC:

Mechums

Run summary

Beaver Creek