search

HARPgroup / HARParchive

This repo houses HARP code development items, resources, and intermediate work products.
1 stars 0 forks source link

SOVA Phase 6 Runoff Integration #167

Open rburghol opened 2 years ago

rburghol commented 2 years ago

Batch to update landuse for each SOVA watershed

filename='/var/www/html/files/vahydro/sova_p532_p6_runoff.txt'
version='p532'
scenario='CFBASE30Y20180615_vadeq'
template=6711736

while IFS= read -r line; do
  #echo "Text read from file: $line"
  read river_hydroid lrseg_hydroid pid vahydro_version <<< "$line"
  echo "Line: $river_hydroid $lrseg_hydroid $pid $version"
  lseg_pid=`drush scr modules/om/src/om_create_landseg.php $river_hydroid $lrseg_hydroid $version $scenario`
  drush scr modules/om/src/om_cbp_lrseg_landuse_file_import.php cmd $lseg_pid $version $scenario 
  # this causes an endless loop now...
  drush scr modules/om/src/om_copy_subcomp.php cmd dh_properties $template dh_properties $lseg_pid landuse_var
done < $filename

SOVA Local Runoff Folders use same Runit data matrix as NOVA

for ro_pid in `cat /var/www/html/files/vahydro/sova_runoff_containers.txt`; do
  drush scr modules/om/src/om_copy_subcomp.php cmd dh_properties 6563947 dh_properties $ro_pid Runit
done

copy (
  select pid
  from dh_properties 
  where propname = '1. Local Runoff Inflows'
  and featureid in (
    select pid from dh_properties 
    where propcode = 'vahydro-1.0'
    and featureid in (
      select hydroid from dh_feature 
      where bundle = 'watershed' and ftype = 'vahydro' 
      and substr(hydrocode,17,2) in ('NR', 'OD', 'OR', 'TU', 'BS', 'MN')
    )
  )
) to '/tmp/sova_runoff_containers.txt' ;

NOVA Land-River runoff elements get updated landuse_var component

copy (
  select * from tmp_p6_lrseg_models 
) to '/tmp/sova_p6_runoff.txt' WITH CSV DELIMITER E'\t';

for ro_pid in `cat /var/www/html/files/vahydro/sova_runoff_containers.txt`; do
  drush scr modules/om/src/om_copy_subcomp.php cmd dh_properties 6711736 dh_properties $ro_pid landuse_var
done
rburghol commented 2 years ago

Testing work flow from: https://github.com/HARPgroup/vahydro/issues/457

Goals:

Run ID Scenario Name
0 CFBASE30Y20180615
4 CFBASE30Y20180615
5 CBASE1808L55CY55R45P50R45P50Y
6 CBASE1808L55CY55R45P10R45P10Y
7 CBASE1808L55CY55R45P90R45P90Y
8 CFBASE30Y20180615
100 CFBASE30Y20180615 

omcbid=343608 # OM elementid for "CBP6 Flows" model object attached to river segment model object 
cbid=4834152 # vahydro pid for "CBP6 Flows" model object attached to river segment model object 
lrseg="A51105_TU3_8880_9230"

drush scr modules/om/src/om_create_pair.php cmd 352129 $omcbid $cbid om_class_cbp_eos_file "$lrseg" prop_feature CBPLandDataConnectionFile 37
drush scr modules/om/src/om_create_pair.php cmd 352129 $omcbid $cbid om_class_cbp_eos_file "$lrseg" prop_feature CBPLandDataConnectionFile 37

drush scr modules/om/src/om.migrate.element.php prop_feature 352127
rburghol commented 2 years ago

Alternate work flow:

rburghol commented 2 years ago

Replace the code below with:

version='p532'
scenario='CFBASE30Y20180615_vadeq'
lseg_pid=`drush scr modules/om/src/om_create_landseg.php 68331 445607 $scenario $version`
drush scr modules/om/src/om_cbp_lrseg_landuse_file_import.php cmd $lseg_pid $version $scenario 

lseg_pid=`drush scr modules/om/src/om_create_landseg.php 68331 445608 $scenario $version`
drush scr modules/om/src/om_cbp_lrseg_landuse_file_import.php cmd $lseg_pid $version $scenario 

lseg_pid=`drush scr modules/om/src/om_create_landseg.php 68331 445670 $scenario $version`
drush scr modules/om/src/om_cbp_lrseg_landuse_file_import.php cmd $lseg_pid $version $scenario

Test in South Fork Roanoke - only use VAHydro now

  1. Get river seg hydroid/model container pid
    • hydroid=68331, pid=4702353, elid=250521
  2. Get the ID of the CBP6 container model (in vahydro)
    • note: for this example the pid = 4831858, and elementid = 342846
    • Find local runoff inflow (LRI) attached to model in no. 1
    • Find CBP6 model attached to LRI model
  3. Get the land segments that flow in this river segment (all phase5 landsegs on sova will get this)
    • This example: A51063_OR1_8280_8020 (445607), A51121_OR1_8280_8020 (445608), and B51161_OR1_8280_8020 (445670)
    • Can use watershed containment view (see VWP Rmd script)
    • set as var $lseg_hydroid = 445607;
  4. Check to see if P6 river segment models exists each land-river segment
    • must be both propcode="vahydro-1.0" and varkey = "om_class_cbp_eos_file"
  5. Create land seg model if missing
    • Pid of model parent of template: 6564010
    • Note: om_copy_subcomp.php wants to know parent pid and sub-comp name, so we look to the CBP6 Runoff template, and grab it's subcomp File-Based Land Segment Runoff Template" -- FWIW its pid = 6711736
    • drush scr modules/om/src/om_copy_subcomp.php cmd dh_properties 6564010 dh_feature 445607 "File-Based Land Segment Runoff Template" 1
    • ex pid: 6735093
  6. Load landseg feature
    • $lseg_feature = entity_load_single('dh_feature', $lseg_hydroid);
  7. Populate attributes:
    • landseg
    • riverseg
    • modelpath: /media/model/p532
  8. Run land use populator script https://github.com/HARPgroup/om/blob/master/drupal/om/src/om_cbp_lrseg_landuse.php
  9. Create a clone of the OM template object attached to the CBP6 Flows object on the riverseg
    • cd /var/www/html/om;
    • New LR elementid
  10. Push landseg to OM as child of Containing Riverseg "CBP6 Runoff"container
    • Load object om_element_connection property
    • Set to "One time Push all Properties" to new LR elementid
    • save()
rburghol commented 2 years ago

To use dev code on deq2: rm -Rf /var/www/html/d.dh/modules/om; ln -s /opt/model/om/drupal/om /var/www/html/d.dh/modules/om

rburghol commented 2 years ago
rburghol commented 2 years ago

Debug: land use area is zero. Check:

rburghol commented 2 years ago

Load them:

#filename='/var/www/html/files/vahydro/sova_p532_p6_runoff-exist.txt'
filename='/var/www/html/files/vahydro/sova_p532_p6_runoff.txt'
version='p532'
scenario='CFBASE30Y20180615_vadeq'
template=6711736

while IFS= read -r line; do
  #echo "Text read from file: $line"
  read river_hydroid lrseg_hydroid pid vahydro_version <<< "$line"
  echo "Line: $river_hydroid $lrseg_hydroid $pid $version"
  lseg_pid=`drush scr modules/om/src/om_create_landseg.php $river_hydroid $lrseg_hydroid $version $scenario`
  drush scr modules/om/src/om_cbp_lrseg_landuse_file_import.php cmd $lseg_pid $version $scenario 
  # this causes an endless loop now...
  drush scr modules/om/src/om_copy_subcomp.php cmd dh_properties $template dh_properties $lseg_pid landuse_var
done < $filename

The supporting queries:

create temp view tmp_p532_lrseg_models as (
  select a.hydroid as rseg_hydroid, 
    b.hydroid as lseg_hydroid,
    c.pid as model_pid, 
    c.propcode as model_version
  from dh_feature as a 
  left outer join dh_feature as b 
  on (
    substr(b.hydrocode,8,13) = substr(a.hydrocode,17,13)
  )
  left outer join dh_properties as c 
  on (
    c.entity_type = 'dh_feature'
    and c.featureid = b.hydroid
    and c.propcode = 'vahydro-1.0'
  )
  where b.ftype = 'cbp532_lrseg'
  and substr(a.hydrocode,17,2) in ('NR', 'OD', 'OR', 'TU', 'BS', 'MN')
  and b.hydroid is not null
  and a.bundle='watershed'
  and b.bundle='landunit'
  and a.ftype='vahydro'
  and (
   NOT (c.propcode = 'vahydro-0.9')
  OR (c.propcode is null)
  )
  and length(a.hydrocode) = 29
  -- uncomment below to sort by either hydrocode
  -- or in a special order to prioritize a given watershed
  order by a.hydrocode
  -- ORDER BY CASE WHEN substr(a.hydrocode,17,1) = 'O' THEN 1 ELSE 2 END
);

copy (
  select * from tmp_p532_lrseg_models 
) to '/tmp/sova_p532_p6_runoff.txt' WITH CSV DELIMITER E'\t';

-- just ones that have models already
copy (
  select * from tmp_p532_lrseg_models 
  where model_version = 'vahydro-1.0'
) to '/tmp/sova_p532_p6_runoff-exist.txt' WITH CSV DELIMITER E'\t';

P6 runoff 


create temp view tmp_p6_lrseg_models as (
  select a.hydroid as rseg_hydroid, 
    b.hydroid as lseg_hydroid,
    c.pid as model_pid, 
    c.propcode as model_version
  from dh_feature as a 
  left outer join dh_feature as b 
  on (
    substr(b.hydrocode,13,13) = substr(a.hydrocode,17,13)
  )
  left outer join dh_properties as c 
  on (
    c.entity_type = 'dh_feature'
    and c.featureid = b.hydroid
    and c.propcode = 'vahydro-1.0'
  )
  where b.ftype = 'cbp6_lrseg'
  and b.hydroid is not null
  and a.bundle='watershed'
  and b.bundle='landunit'
  and a.ftype='vahydro'
  and (
   NOT (c.propcode = 'vahydro-0.9')
  OR (c.propcode is null)
  )
  and length(a.hydrocode) = 29
  order by a.hydrocode
);

copy (
  select * from tmp_p6_lrseg_models 
) to '/tmp/sova_p6_runoff.txt' WITH CSV DELIMITER E'\t';
rburghol commented 2 years ago 
filename='/var/www/html/files/vahydro/sova_p532_p6_runoff-O-1st.txt'

while IFS= read -r line; do
  #echo "Text read from file: $line"
  read river_hydroid lrseg_hydroid pid vahydro_version <<< "$line"
  echo "Line: $river_hydroid $lrseg_hydroid $pid $version"
  drush scr modules/om/src/om_saveprop.php pid $pid
done < $filename
rburghol commented 2 years ago

Find runoff container with the erroneous variable landuse_var (they shouldn't have one) and delete that property.

filename="/var/www/html/files/vahydro/sova_runoff_containers_with_badvar.txt"
for i in `cat $filename`; do
  drush scr modules/om/src/om_deleteprop.php pid $i
done

The file:

copy (
  select b.pid
  from dh_properties as a 
  left outer join dh_properties as b
  on (
    b.featureid = a.pid
    and b.entity_type = 'dh_properties'
    and b.propname = 'landuse_var'
  )
  where a.propname = '1. Local Runoff Inflows'
    and b.pid is not null
) to '/tmp/sova_runoff_containers_with_badvar.txt' ;