Test VAHydro hydrography in cbp/hsp2

HARPgroup / HARParchive

This repo houses HARP code development items, resources, and intermediate work products.

1 stars 0 forks source link

Test VAHydro hydrography in cbp/hsp2 #738

Open rburghol opened 1 year ago

rburghol commented 1 year ago

Overview

subsheds, cbp-6.1

Scripts for achieving this are here: https://github.com/HARPgroup/cbp_wsm/issues/85
https://github.com/HARPgroup/HARParchive/issues/546

Tasks:

[x] Verify that all subwatersheds are created and FTABLEd
- [x] Hydrocodes #761
- [x] Tidal #747
- [x] Come up with a solid work-flow to create FTABLE, get FTABLE file name, get new segment ID, add model, save FTABLE as new name, update hydrocode in subwatershed feature
- [x] Test small headwater (North Anna river
- [x] Test medium tidal-fresh ending basin (Chickahominy, see comments below)
- [x] Test HUC (James)
[x] configure scenario subsheds uses hsp2 model and cbp-6.1
[x] Verify cbp-6.1 models #770
- [x] Does subsheds get the right land runoff values, i.e., can land handle that given that there is a dependence on GEO in land simulation QA?
[x] Run full watersheds, check analysis against VAHydro and cbp-6.0

Workflow for Updating sub-watershed hydrocode during resegmentation.

Run add_sub_watershed.bat
Change feature hydrocode
Change vahydro-1.0 riverseg prop

rburghol commented 1 year ago

York/North Anna

basin=YP3_6670_6720
scenario=subsheds

rburghol commented 1 year ago

Chickahominy

Outlet is JB0_7050_0000
Does a 0000 watershed work ok in basingen.csh?

Does the Diascund Creek get included since it is a _0001 segment?

No. Nor does the Chickahominy mainstem, since it also is an 0001 segment.

Therefore, I created a merged basin with the land and river segments from the Chick, Disacund Creek, and the tidal segments that flow into 7050 as chick group.


basin=chick
scenario=subsheds
met_scen=`cbp get_config $scenario land MET` 
#Check the met WDM status:
cbp check_met $met_scen $basin
#Get the prad dataset for the land (or river) simulation of interest
prad_scen=`cbp get_config $scenario land "PRECIP ATMOS DEPOSITION"` 
#Check the met WDM status:
cbp check_prad $prad_scen $basin
# set ENV variables
. hspf_config
# run the land 
segs=`cbp get_landsegs $basin`
read -r scope land_scenario < <(cbp get_config $scenario river "LAND SCENARIO")
for i in $segs; do
echo "sbatch /opt/model/meta_model/run_model hspf_cbp6 $land_scenario $i auto land "
sbatch /opt/model/meta_model/run_model hspf_cbp6 $land_scenario $i auto land 
done
segs=`cbp get_riversegs $basin`
for i in $segs; do
/opt/model/meta_model/run_model hsp2_cbp6 $scenario $i auto river 
done

rburghol commented 1 year ago

All of James:

basin=JL7_7070_0001
scenario=subsheds
met_scen=`cbp get_config $scenario land MET` 
#Check the met WDM status:
cbp check_met $met_scen $basin
#Get the prad dataset for the land (or river) simulation of interest
prad_scen=`cbp get_config $scenario land "PRECIP ATMOS DEPOSITION"` 
#Check the met WDM status:
cbp check_prad $prad_scen $basin
# set ENV variables
. hspf_config
# run the land 
segs=`cbp get_landsegs $basin`
read -r scope land_scenario < <(cbp get_config $scenario river "LAND SCENARIO")
for i in $segs; do
  echo "sbatch /opt/model/meta_model/run_model hspf_cbp6 $land_scenario $i auto land "
  sbatch /opt/model/meta_model/run_model hspf_cbp6 $land_scenario $i auto land 
done
segs=`cbp get_riversegs $basin`
for i in $segs; do
  /opt/model/meta_model/run_model hsp2_cbp6 $scenario $i auto river 
done

rburghol commented 1 year ago

Rappahannock:

[x] verify White Run is set up properly ( I think it is not)

[x] Fix other errors: in

[x] RU3_6170_6040: Problem opening wdm for upstream segment RU2_6220_6170 WDM does not exist
[x] RU4_5640_0003: ...RU4_5642_5640 WDM does not exist
[x] RU4_6040_6030: ...RU3_6170_6040 WDM does not exist

[x] RU5_6030_0001: RU4_5640_6030 does not exist

./run/resegment/add_sub_watershed.bat RU2_6090_6220_whites_run RU2_6090_6220 cbp-6.1 subsheds 0.9 
riverseg=RU2_6091_6090
Rscript run/resegment/ftable_creation.R $riverseg manual vahydro-1.0 "/opt/model/p6/vadeq/input/param/river/vahydro_2022/ftables/" 1.13 8395.2 3 0.01540171
mkdir tmp/scratch/rwb
/opt/model/meta_model/run_model hsp2_cbp6 $scenario $riverseg tmp/scratch/rwb river prep 01_run_rug

[ ]


. hspf_config
basin=RU5_6030_0001
scenario=subsheds
met_scen=`cbp get_config $scenario land MET` 
cbp check_met $met_scen $basin
prad_scen=`cbp get_config $scenario land "PRECIP ATMOS DEPOSITION"` 
cbp check_prad $prad_scen $basin
scenario="subsheds"
# run land
segs=`cbp get_landsegs $basin`
location="p6_vadeq" # crucial to avoid collisions if running multiple installs
read -r scope land_scenario < <(cbp get_config $scenario river "LAND SCENARIO")
for i in $segs; do
job_name="${location}_${scenario}_${i}"
echo "sbatch --job-name=\"$job_name\" /opt/model/meta_model/run_model hspf_cbp6 $land_scenario $i auto land "
sbatch --job-name="$job_name" /opt/model/meta_model/run_model hspf_cbp6 $land_scenario $i auto land 
done
# run river
# can now immediately be submitted after submitting land 
# since the dependency ordering works with land and river 
segs=`cbp get_riversegs $basin`
for i in $segs; do
deps=`cbp mm_river_deps $scenario $i`
echo "Segment $i Found deps= $deps"
job_name=`cbp mm_job_name $scenario $i`
echo "sbatch --job-name=$job_name $deps /opt/model/meta_model/run_model hsp2_cbp6 $scenario $i  auto river"
sbatch --job-name=$job_name $deps /opt/model/meta_model/run_model hsp2_cbp6 $scenario $i auto river
done

rburghol commented 1 year ago

Rappahannock:

[ ] verify White Run is set up properly ( I think it is not)

./run/resegment/add_sub_watershed.bat RU2_6090_6220_whites_run RU2_6090_6220 cbp-6.1 subsheds 0.9 
riverseg=RU2_6091_6090
Rscript run/resegment/ftable_creation.R $riverseg manual vahydro-1.0 "/opt/model/p6/vadeq/input/param/river/vahydro_2022/ftables/" 1.13 8395.2 3 0.01540171
mkdir tmp/scratch/rwb
/opt/model/meta_model/run_model hsp2_cbp6 $scenario $riverseg tmp/scratch/rwb river prep 01_run_rug

[ ]


. hspf_config
basin=RU5_6030_0001
scenario=subsheds
met_scen=`cbp get_config $scenario land MET` 
cbp check_met $met_scen $basin
prad_scen=`cbp get_config $scenario land "PRECIP ATMOS DEPOSITION"` 
cbp check_prad $prad_scen $basin
segs=`cbp get_riversegs $basin`
scenario="subsheds"
# run land
segs=`cbp get_landsegs $basin`
location="p6_vadeq" # crucial to avoid collisions if running multiple installs
read -r scope land_scenario < <(cbp get_config $scenario river "LAND SCENARIO")
for i in $segs; do
job_name="${location}_${scenario}_${i}"
echo "sbatch --job-name=\"$job_name\" /opt/model/meta_model/run_model hspf_cbp6 $land_scenario $i auto land "
sbatch --job-name="$job_name" /opt/model/meta_model/run_model hspf_cbp6 $land_scenario $i auto land 
done
# run river
# can now immediately be submitted after submitting land 
# since the dependency ordering works with land and river 
for i in $segs; do
deps=`cbp mm_river_deps $scenario $i`
echo "Segment $i Found deps= $deps"
job_name=`cbp mm_job_name $scenario $i`
echo "sbatch --job-name=$job_name $deps /opt/model/meta_model/run_model hsp2_cbp6 $scenario $i  auto river"
sbatch --job-name=$job_name $deps /opt/model/meta_model/run_model hsp2_cbp6 $scenario $i auto river
done

rburghol commented 1 year ago

Run the SF Shenandoah River in its entirety. It has not previously completed when ding the PM model?

Previous full basin run failed, but
running the last 3 segment of SF Shenandoah above confluence worked finE
- segs="PS4_5840_5240 PS5_5240_5200 PS5_5200_4380"
Try running the p6ss version to test:
- scenario=p6ss
- basin=PS5_5200_4380
- monitor with https://deq1.bse.vt.edu/cgi-bin/mapserv?map=/var/www/html/mapserv/vahydro_result.map&propname=Qout&model_version=cbp-6.1&runid=p6ss&layers=model_run_status&bundle=watershed&ftype=vahydro&layers=minor_basins
```
basin=PS5_5200_4380
# new flexible basingen 
cbp basingen $basin
segs=`cbp get_riversegs $basin`
```

for i in $segs; do deps=cbp mm_river_deps $scenario $i echo "Segment $i Found deps= $deps" job_name=cbp mm_job_name $scenario $i echo "sbatch --job-name=$job_name $deps /opt/model/meta_model/run_model hsp2_cbp6 $scenario $i auto river" sbatch --job-name=$job_name $deps /opt/model/meta_model/run_model hsp2_cbp6 $scenario $i auto river done

- Zero flow out in `PM1_4252_4250` cause failure.
- See code:

hydr<-read.csv('/media/model/p6/out/river/subsheds/hydr/PM1_4252_4250_hydr.csv') hydr[1:2,]

- initial volume was `10000`, which was outside of the FTABLE `input/param/river/vahydro_2022/HYDR.csv`
- tried setting to `100` (just guessing), and that was waaay into the zero flow part of the ftable.
- Examined ftable and set to 1500, a flow out of about 1-2 cfs: http://deq1.bse.vt.edu:81/p6/vadeq/tmp/uci/river/subsheds/PM1_4252_4250.uci

rburghol commented 1 year ago

Final Potomac.
Fails 3 segments above the fall line: PM1_4250_4500 (Great Falls)

[ ] Long Branch Clopper Lake (Used to be Seneca in p6): ows-watershed-dash-info/454129
- First 6-7 months have zero outflows for much of the time till things go normal.
- check initial conditions? or ftable? or both?
- initial conditions of VOL were way too high, outside of ftable, causing errors and zero outflow.
Also, Difficult Run has not been converted, so it isn't likely to run properly: ows-watershed-dash-info/477835 (Difficult Run - see this issue: https://github.com/HARPgroup/vahydro/issues/392)
```
ERROR:  the wdm file
../../../tmp/wdm/river/subsheds/stream/PM1_4252_4250.wdm
Had zero flow for the first month
```

rburghol commented 1 year ago

Mattaponi

cbp check_land $scenario $basin
Land segments with at least one LU missing:
N51033 N51097 N51137 N51177 N51101

Need to rerun land

basin=YM4_6620_0001
scenario="subsheds"
# run land
segs=`cbp get_landsegs $basin`
location="p6_vadeq" # crucial to avoid collisions if running multiple installs
read -r scope land_scenario < <(cbp get_config $scenario river "LAND SCENARIO")
for i in $segs; do
job_name="${location}_${scenario}_${i}"
echo "sbatch --job-name=\"$job_name\" /opt/model/meta_model/run_model hspf_cbp6 $land_scenario $i auto land "
sbatch --job-name="$job_name" /opt/model/meta_model/run_model hspf_cbp6 $land_scenario $i auto land 
done
# run river
# can now immediately be submitted after submitting land 
# since the dependency ordering works with land and river 
segs=`cbp get_riversegs $basin`
for i in $segs; do
deps=`cbp mm_river_deps $scenario $i`
echo "Segment $i Found deps= $deps"
job_name=`cbp mm_job_name $scenario $i`
echo "sbatch --job-name=$job_name $deps /opt/model/meta_model/run_model hsp2_cbp6 $scenario $i  auto river"
sbatch --job-name=$job_name $deps /opt/model/meta_model/run_model hsp2_cbp6 $scenario $i auto river
done

rburghol commented 1 year ago

Ragged Mountain was lingering (not old model has RM as child of SF reservoir, but that is not correct hydrography (altho it works via remote links), so set this up with actual parentage and will later figure out how to organize those in the final model):

./run/resegment/add_sub_watershed.bat JL4_6520_6710_ragged_mtn JL4_6520_6710 cbp-6.1 subsheds 1.8
Rscript run/resegment/ftable_creation.R JL4_6521_6520 local_channel vahydro-1.0 "/opt/model/p6/vadeq/input/param/river/vahydro_2022/ftables/"
Edited HYDR to avoid initial volume craziness (set to 0.27 based on ftable): JL4_6521_6520,1,26.83,52.48,NA,0.5,NA,0.27,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA

rburghol commented 1 year ago

Sandy River reservoir:

./run/resegment/add_sub_watershed.bat JA1_7640_7280_sandy_river JA1_7640_7280 cbp-6.1 subsheds 31.52
Find NHD info:
- Set out point out_point = sf::st_sfc(sf::st_point(c(-78.3179, 37.2586)), crs = 4326) nhd_out <- get_nhdplus(out_point)
- length_ft, da_sqmi, cslope 36748.68884 31.5178436982382 0.00140342204952214
- Run te rest of the script
Create ftable: Rscript run/resegment/ftable_creation.R JA1_7641_7640 manual vahydro-1.0 "/opt/model/p6/vadeq/input/param/river/vahydro_2022/ftables/" 31.52 36748.69 2 0.0014
set intial volume to 5.0 (*note: must re-set initial volumes when running as impoundment with new ftable)

rburghol commented 1 year ago

Swift Creek ( does not get run when running appomattox to segment 0001 )

. hspf_config
basin=JA2_7290_0001
scenario="subsheds"
# run land
segs=`cbp get_landsegs $basin`
location="p6_vadeq" # crucial to avoid collisions if running multiple installs
read -r scope land_scenario < <(cbp get_config $scenario river "LAND SCENARIO")
for i in $segs; do
  job_name="${location}_${scenario}_${i}"
  echo "sbatch --job-name=\"$job_name\" /opt/model/meta_model/run_model hspf_cbp6 $land_scenario $i auto land "
  sbatch --job-name="$job_name" /opt/model/meta_model/run_model hspf_cbp6 $land_scenario $i auto land 
done
# run river
# can now immediately be submitted after submitting land 
# since the dependency ordering works with land and river 
segs=`cbp get_riversegs $basin`
for i in $segs; do
  deps=`cbp mm_river_deps $scenario $i`
  echo "Segment $i Found deps= $deps"
  job_name=`cbp mm_job_name $scenario $i`
  echo "sbatch --job-name=$job_name $deps /opt/model/meta_model/run_model hsp2_cbp6 $scenario $i  auto river"
  sbatch --job-name=$job_name $deps /opt/model/meta_model/run_model hsp2_cbp6 $scenario $i auto river
done