search

HARPgroup / HARParchive

This repo houses HARP code development items, resources, and intermediate work products.
1 stars 0 forks source link

Test base cbp/hsp2 model #737

Open rburghol opened 1 year ago

rburghol commented 1 year ago

Checklist for basins that have been run:

model version = cbp-6.0 runid = hsp2_2022

Monitor progress:

hsp2_2022, cbp-6.0 This model uses the baseline CBP hydrography, northern rivers only, so no custom vahydro sub-watersheds, nor southern rivers segments will appear in these model simulations.

FTYPE:

Case Studies

jdkleiner commented 1 year ago

James:

Model_Map_vahydro-1 0runid_11_cbp-6 0hsp2_2022_james

Potomac:

Model_Map_vahydro-1 0runid_11_cbp-6 0hsp2_2022_potomac

All:

Model_Map_vahydro-1 0runid_11_cbp-6 0hsp2_2022_all

jdkleiner commented 1 year ago

Regenerated James Map 2/15/23: Model_Map_vahydro-1 0runid_11_cbp-6 0hsp2_2022_james

jdkleiner commented 1 year ago

3/8/23

Model_Map_vahydrovahydro-1 0runid_11_cbp60cbp-6 0hsp2_2022_potomac

jdkleiner commented 1 year ago

3/8/23 Potomac Run Testing

cbp check_land $rscenario $rseg Land segments with at least one LU missing: N11001 N24033 N51510 N51610 N24017 N51179 N24037 N51099 N51193 N51133 N51013 N54031 N51187 N51043 N51840 N51660 N42041 N54003 N54037 N51139 L51015 N51790 N42099 N54065 N42057 H54031 N54027 N24001 N42009 H51139 H54023 L54023 N54023 N54057 L54031 N42111 N24023 H24023 N54077 N54093 H54057

- land simulation: 
   - kicked off ~10:20am
   - completed ~11:47am
- river simulation:
   - `cat /opt/model/p6/vadeq/config/control/script/hsp2_2022.con`
      - ensure this file is setting `MM_MODEL`
   - `cat /opt/model/p6/vadeq/run/mm/mm_run_basin`
      - this can be used to run full basins
   - run the river sim for all of Potomac:
      - `nohup mm_run_basin $rscenario $rseg &`
- Run test of 3 segs:
   - Seneca Creek: rseg=PU1_5520_5210 (om-model-info/7020978)
      - Potomac River South Branch North Fork: rseg=PU3_5210_5050 (om-model-info/7023098)

jdk@deq2:/opt/model/p6/vadeq$ echo $rseg PU3_5210_5050 jdk@deq2:/opt/model/p6/vadeq$ allsegs=cbp get_riversegs $rseg Loading /opt/model/p6/vadeq/hspf.config jdk@deq2:/opt/model/p6/vadeq$ echo $allsegs PU1_5520_5210 PU2_5700_5210 PU3_5210_5050

jdk@deq2: cd /opt/model/p6/vadeq cat run/mm/mm_run_basin

set up model environment

MODEL_ROOT=/opt/model/p6/vadeq META_MODEL_ROOT=/opt/model/meta_model IGNORE_PROBLEMS=0 export MODEL_ROOT META_MODEL_ROOT IGNORE_PROBLEMS

input arguments

rseg=PU3_5210_5050 # has one seg above it rmodel=hsp2_cbp6 rscenario=hsp2_2022 lmodel=hspf_cbp6 read -r scope lscenario < <(cbp get_config $rscenario river "LAND SCENARIO")

check meteorology status

metscenario=cbp get_config $rscenario land MET cbp check_met $metscenario $rseg cbp check_land $rscenario $rseg

cbp basingen.csh $rscenario ${rseg:4:4} # needed if you get message "No such file or directory"

run the land simulation (sbatch uses slurm to run land simultaneously)

lsegs=cbp get_landsegs $rseg for i in $lsegs; do echo "sbatch $META_MODEL_ROOT/run_model $lmodel $lscenario $i auto land" sbatch $META_MODEL_ROOT/run_model $lmodel $lscenario $i auto land done

run the river simulation

may need to remove the nohup file before starting

jkleiner@deq2:/opt/model/p6/vadeq$ rm nohup.out

rm: remove write-protected regular file 'nohup.out'? Y

nohup run/mm/mm_run_basin $rscenario $rseg & tail -f nohup.out Ctrl + C # to close

jdkleiner commented 1 year ago

3/9/23 Kick-off full potomac run

jdk@deq2:
cd /opt/model/p6/vadeq
# cat run/mm/mm_run_basin

# set up model environment
MODEL_ROOT=/opt/model/p6/vadeq
META_MODEL_ROOT=/opt/model/meta_model
IGNORE_PROBLEMS=0
export MODEL_ROOT META_MODEL_ROOT IGNORE_PROBLEMS

# input arguments
rseg=P
rmodel=hsp2_cbp6
rscenario=hsp2_2022
lmodel=hspf_cbp6
read -r scope lscenario < <(cbp get_config $rscenario river "LAND SCENARIO")

# check meteorology status
metscenario=`cbp get_config $rscenario land MET`
cbp check_met $metscenario $rseg 
cbp check_land $rscenario $rseg

# reran land (kicked off ~4:38 3/8/23)
# likely did not need to rerun land
# run the land simulation (sbatch uses slurm to run land simultaneously)
lsegs=`cbp get_landsegs $rseg`
for i in $lsegs; do
  echo "sbatch $META_MODEL_ROOT/run_model $lmodel $lscenario $i auto land"
  sbatch $META_MODEL_ROOT/run_model $lmodel $lscenario $i auto land 
done

# run the river simulation
nohup run/mm/mm_run_basin $rscenario $rseg &
tail -f nohup.out
Ctrl + C # to close

Not all Potomac Segments ran:

Model_Map_vahydro-vahydro-1 0-runid_11__cbp60-cbp-6 0-hsp2_2022_potomac

jdkleiner commented 1 year ago

3/13/23 York Pamunkey

select * from dh_feature where bundle = 'watershed' and ftype ilike '%cbp60%' and hydrocode like 'YP%_0001';
 hydroid |   hydrocode   | ftype |  bundle   |       name        | status | module |         modified          | fstatus | uid
---------+---------------+-------+-----------+-------------------+--------+--------+---------------------------+---------+-----
  606097 | YP0_6840_0001 | cbp60 | watershed | Totoponomoy Creek |      1 |        | 2023-02-16 13:23:07.03359 |         |   0
  606108 | YP4_6750_0001 | cbp60 | watershed | Pamunkey River    |      1 |        | 2023-02-16 13:23:07.14021 |         |   0
(2 rows)
cd /opt/model/p6/vadeq

# set up model environment
MODEL_ROOT=/opt/model/p6/vadeq
META_MODEL_ROOT=/opt/model/meta_model
IGNORE_PROBLEMS=0
export MODEL_ROOT META_MODEL_ROOT IGNORE_PROBLEMS

# input arguments
rseg=YP4_6750_0001
rmodel=hsp2_cbp6
rscenario=hsp2_2022
lmodel=hspf_cbp6
read -r scope lscenario < <(cbp get_config $rscenario river "LAND SCENARIO")

# check meteorology status
metscenario=`cbp get_config $rscenario land MET`
cbp check_met $metscenario $rseg 
   Loading /opt/model/p6/vadeq/hspf.config
   Loading /opt/model/p6/vadeq/hspf.config
   Loading /opt/model/p6/vadeq/hspf.config
   Looking for land segments: N51085 N51109 N51003 N51065 N51075 N51137 N51177 N51033 N51101
   Could not find /opt/model/p6/vadeq/input/scenario/climate/met/nldas1221/met_N51101.wdm
   Found 9 land-segments
   Found 8 land-segments with met WDMs
   Missing 1 land-segment met:
   Missing land segment-landuse combos:
   N51101.wdm

# Prepared meteorology for missing land seg
prad_scen=`cbp get_config $rscenario land "PRECIP ATMOS DEPOSITION"` 
cbp check_prad $prad_scen $rseg
wdm_pm_one N51101 1984010100 2020123123 nldas2 harp2021 $metscenario $prad_scen

# Check again
cbp check_met $metscenario $rseg
   Loading /opt/model/p6/vadeq/hspf.config
   Loading /opt/model/p6/vadeq/hspf.config
   Loading /opt/model/p6/vadeq/hspf.config
   Looking for land segments: N51085 N51109 N51003 N51065 N51075 N51137 N51177 N51033 N51101
   Found 9 land-segments
   Found 9 land-segments with met WDMs
   Missing 0 land-segment met:
   Missing land segment-landuse combos:
cbp check_land $rscenario $rseg
# cbp basingen.csh $rscenario ${rseg:4:4} # needed if you get message "No such file or directory"

# run the land simulation (sbatch uses slurm to run land simultaneously)
lsegs=`cbp get_landsegs $rseg`
for i in $lsegs; do
  echo "sbatch $META_MODEL_ROOT/run_model $lmodel $lscenario $i auto land"
  sbatch $META_MODEL_ROOT/run_model $lmodel $lscenario $i auto land 
done

# run the river simulation
rsegs=`cbp get_riversegs $rseg`
for i in $rsegs; do
  # echo "$META_MODEL_ROOT/run_model $rmodel $rscenario $i auto river"
  $META_MODEL_ROOT/run_model $rmodel $rscenario $i auto river
done
rseg=YP0_6840_0001

Runs successful (all white segs on this map are tidal)

Model_Map_vahydro-vahydro-1 0-runid_11__cbp60-cbp-6 0-hsp2_2022_york pamunkey

York Mattaponi

rseg=YM4_6620_0001

Model_Map_vahydro-vahydro-1 0-runid_11__cbp60-cbp-6 0-hsp2_2022_york mattaponi

Rappahannock

 hydroid |   hydrocode   | ftype |  bundle   |        name        | status | module |          modified          | fstatus | uid
---------+---------------+-------+-----------+--------------------+--------+--------+----------------------------+---------+-----
  605667 | RL0_6540_0001 | cbp60 | watershed | Piscataway Creek   |      1 |        | 2023-02-16 13:22:57.615118 |         |   0
  605668 | RL1_6180_0001 | cbp60 | watershed | Cat Point Creek    |      1 |        | 2023-02-16 13:22:57.623768 |         |   0
  605686 | RU5_6030_0001 | cbp60 | watershed | Rappahannock River |      1 |        | 2023-02-16 13:22:58.103663 |         |   0
(3 rows)
rseg=RU5_6030_0001

ran into error: 

Running: /opt/model/meta_model/models/hsp2_cbp6/river/run/03_upstream hsp2_2022 RU5_6030_0001 /opt/model/p6/vadeq/tmp/scratch/18768
Loading model configuration
MODEL_VERSION_CODE = cbp-6.0
Loading /opt/model/p6/vadeq/hspf.config
Searching for TIME END TIME in /opt/model/p6/vadeq/config/control/river/hsp2_2022.con
Found model timespan 1984 to 2020
model_config finished.
Copying river WDM /opt/model/p6/vadeq/tmp/wdm/river/hsp2_2022/eos/RU5_6030_0001'.wdm' RU5_6030_0001'.wdm'
'/opt/model/p6/vadeq/tmp/wdm/river/hsp2_2022/eos/RU5_6030_0001.wdm' -> 'RU5_6030_0001.wdm'
 Making stream wdm for segment RU5_6030_0001
 Scenario hsp2_2022

 upstream segment RU4_5640_6030Problem adding upstream segments to wdm file, segment ,RU5_6030_0001

 PROBLEM FILE WRITTEN

Problem opening wdm for upstream segment RU4_5640_6030
  WDM does not exist
 upstream segment probably not run

Error: Problem found in step hsp2_cbp6 => river / run / 03_upstream
       See /opt/model/p6/vadeq/tmp/scratch/18768/problem
jdkleiner commented 1 year ago

3/14/23 Re-run Potomac

rseg=PM7_4820_0001

# run the river simulation
nohup run/mm/mm_run_basin $rscenario $rseg &
less +F nohup.out
# Ctrl + C to navigate around the log file
# F to view the log in real time again
# q to exit

Model_Map_vahydro-vahydro-1 0-runid_11__cbp60-cbp-6 0-hsp2_2022_Qout_potomac

jdkleiner commented 1 year ago

3/15/23 Run South Fork Shenandoah (with slurm)

rseg=PS5_5200_4380
rmodel=hsp2_cbp6
rscenario=hsp2_2022
rsegs=`cbp get_riversegs $rseg`
for i in $rsegs; do
  deps=`cbp mm_river_deps $rscenario $i`
  echo "Segment $i Found deps= $deps"
  job_name=`cbp mm_job_name $rscenario $i`
  echo "sbatch --job-name=$job_name $deps /opt/model/meta_model/run_model $rmodel $rscenario $i  auto river"
  sbatch --job-name=$job_name $deps /opt/model/meta_model/run_model $rmodel $rscenario $i auto river
done

ERROR: the wdm file ../../../tmp/wdm/river/hsp2_2022/stream/PS3_6161_6280.wdm Had zero flow for the first month

- https://github.com/HARPgroup/HARParchive/issues/771#issuecomment-1470298647

### 3/17/23 Rerun South Fork Shenandoah

- downstream seg:

cd /opt/model/p6/vadeq

input arguments

rseg=PS3_6280_6230 rmodel=hsp2_cbp6 rscenario=hsp2_2022 lmodel=hspf_cbp6

Check the met WDM status:

metscenario=cbp get_config $rscenario land MET cbp check_met $metscenario $rseg

cbp basingen.csh $rscenario ${rseg:4:4} # needed if you get message "No such file or directory"

Prepared meteorology for any missing land segments

prad_scen=cbp get_config $rscenario land "PRECIP ATMOS DEPOSITION" cbp check_prad $prad_scen $rseg

wdm_pm_one N51101 1984010100 2020123123 nldas2 harp2021 $metscenario $prad_scen

Set ENV variables

. hspf_config

run the land simulation (sbatch uses slurm to run land simultaneously)

lsegs=cbp get_landsegs $rseg read -r scope lscenario < <(cbp get_config $rscenario river "LAND SCENARIO") for i in $lsegs; do echo "sbatch /opt/model/meta_model/run_model $lmodel $lscenario $i auto land " sbatch /opt/model/meta_model/run_model $lmodel $lscenario $i auto land done

run the river simulation (for that single segment)

$META_MODEL_ROOT/run_model $rmodel $rscenario $rseg auto river


- upstream seg:

rseg=PS3_6161_6280

run the river simulation

nohup $META_MODEL_ROOT/run_model $rmodel $rscenario $rseg auto river &

jdkleiner commented 1 year ago

3/15/23 Run James

# input arguments
rseg=JL7_7070_0001
rmodel=hsp2_cbp6
rscenario=hsp2_2022
lmodel=hspf_cbp6

# Check the met WDM status:
metscenario=`cbp get_config $rscenario land MET` 
cbp check_met $metscenario $rseg
# cbp basingen.csh $rscenario ${rseg:4:4} # needed if you get message "No such file or directory"

# Prepared meteorology for any missing land segments
prad_scen=`cbp get_config $rscenario land "PRECIP ATMOS DEPOSITION"` 
cbp check_prad $prad_scen $rseg
# wdm_pm_one N51101 1984010100 2020123123 nldas2 harp2021 $metscenario $prad_scen

# Set ENV variables
. hspf_config

# run the land simulation (sbatch uses slurm to run land simultaneously)
lsegs=`cbp get_landsegs $rseg`
read -r scope lscenario < <(cbp get_config $rscenario river "LAND SCENARIO")
for i in $lsegs; do
  echo "sbatch /opt/model/meta_model/run_model $lmodel $lscenario $i auto land "
  sbatch /opt/model/meta_model/run_model $lmodel $lscenario $i auto land 
done

# run the river simulation (using slurm)
rsegs=`cbp get_riversegs $rseg`

for i in $rsegs; do
  deps=`cbp mm_river_deps $rscenario $i`
  echo "Segment $i Found deps= $deps"
  job_name=`cbp mm_job_name $rscenario $i`
  echo "sbatch --job-name=$job_name $deps $META_MODEL_ROOT/run_model $rmodel $rscenario $i  auto river"
  sbatch --job-name=$job_name $deps $META_MODEL_ROOT/run_model $rmodel $rscenario $i auto river
done
jdkleiner commented 1 year ago

3/16/23

Run Appomattox

rsegsA=`cbp get_riversegs JA5_7480_0001`
rsegsB=`cbp get_riversegs JA2_7290_0001`
rsegs=$rsegsA" "$rsegsB

Run Small Tribs

# land sim
lsegsA=`cbp get_landsegs JB3_7053_0001`
lsegsB=`cbp get_landsegs JB2_7800_0001`
lsegsC=`cbp get_landsegs JB1_8090_0001`
lsegsD=`cbp get_landsegs JB0_7052_0001`
lsegsE=`cbp get_landsegs JB0_7051_0001`

# concatenate land segs 
lsegs=$lsegsA" "$lsegsB" "$lsegsC" "$lsegsD" "$lsegsE
# remove any duplicate land segs from string
read lsegs <<< $( echo "$lsegs" | awk '{for (i=1;i<=NF;i++) if (!a[$i]++) printf("%s%s",$i,FS)}{printf("\n")}' )

# river sim 
rsegsA=`cbp get_riversegs JB3_7053_0001`
rsegsB=`cbp get_riversegs JB2_7800_0001`
rsegsC=`cbp get_riversegs JB1_8090_0001`
rsegsD=`cbp get_riversegs JB0_7052_0001`
rsegsE=`cbp get_riversegs JB0_7051_0001`
rsegs=$rsegsA" "$rsegsB" "$rsegsC" "$rsegsD" "$rsegsE