Harvest river segment evap and precip data

[rburghol]

I want to harvest river segment WDM data for:

• WDM1 1000 EVAP
• WDM2 2000 HPRC

For example, this river segment output /opt/model/p6/p6_gb604/tmp/wdm/river/CFBASE30Y20180615/stream/PS2_5560_5100.wdm

I'd like to put both of them in a single CSV (like we've done for other things). I know we have something that "sorta" does that, just looking for some guidance on what is the latest greatest and how you might apply it.

[hdaniel7]

After digging into some of the old "openmi-om" scripts a bit, I think I found one which should complete this task with some minor modifications -- I'll work on that for awhile and get back to you with updates.

[hdaniel7]

Rob -- are you sure that 1000 and 2000 are the correct DSNs (or that these .wdm files contain them)? I was running into some issues with my scripts and narrowed it down to quick_wdm_2_txt -- when running it with DSNs such as 0111, quick_wdm_2_txt works perfectly, as shown in the following image.

When doing the same for DSN 1000, the following problem file is generated:

[rburghol]

Well, no I am not sure :). Actually, I am sure about the DSNs, but now that you point it out WDM1 and WDM2 (see above in issue summary for which is which) that supply that data are not the river segment WDMs -- rather, they are the land segment WDMs. Do we have those WDMs??

[hdaniel7]

I don't believe we do have the land segment WDMs -- from my knowledge the only WDMs we have for the land segments are the land-use separated WDMs, such as aopH24021.wdm, which are stored in locations such as /opt/model/p6/p6_gb604/tmp/wdm/land/aop/CFBASE30Y20180615.

[rburghol]

Daniel -- I believe that I already requested and received the climate WDMs from Gopal. There are a number of files he delivered, and they are referenced in the river UCI files as follows:

WDM1       21   ../../../input/scenario/climate/met/N20150521J96/met_N42071.wdm
WDM2       22   ../../../input/scenario/climate/prad/PRC20170329/prad_N42071.wdm

From the UCI it looks that "met" files hold the ET (dsn 1000), and "prad" files hold the precip (dsn 2000). There seems to be 2 met data sets: "5545HS50CA2" and "N20150521J96" and 1 prad data set "5545KK50AA". Interestingly, there are some met and some prad files with river segment prefixes, but the conventiojn in the past was to use county level met/prad for riverine ET/PRECIP.

Here are some sample files to examine:

/opt/model/p6/p6_gb604/input/scenario/climate/met/5545HS50CA2/met_N51157.wdm
/opt/model/p6/p6_gb604/input/scenario/climate/met/N20150521J96/met_N51007.wdm
/opt/model/p6/p6_gb604/input/scenario/climate/prad/5545KK50AA/prad_N51840.wdm
/opt/model/p6/p6_gb604/input/scenario/climate/prad/5545KK50AA/prad_N51820.wdm

/opt/model/p6/p6_gb604/input/scenario/climate/met/N20150521J96/met_JL6_6960_6970.wdm
/opt/model/p6/p6_gb604/input/scenario/climate/prad/PRC20170731/prad_YP3_6670_6720.wdm

And some export stats:

rob@deq2:/opt/model/p6/p6_gb604/input/scenario/climate/prad/PRC20170731$ /opt/model/p6-devel/p6-4.2018/code/bin/quick_wdm_2_txt_hour_2_hour
  program to write hourly ascii output from a wdm
 wdm name, start year, end year, dsn
prad_N51177.wdm,1984,2014,2000
 hourly average =    4.8991177271506823E-003
 annual average =    42.946614212537170

rob@deq2:/opt/model/p6/p6_gb604/input/scenario/climate/met/N20150521J96$ /opt/model/p6-devel/p6-4.2018/code/bin/quick_wdm_2_     txt_hour_2_hour
  program to write hourly ascii output from a wdm
 wdm name, start year, end year, dsn
met_N51007.wdm,1984,2014,1000
 hourly average =    3.7574039555417229E-003
 annual average =    32.938130313754009

[hdaniel7]

Alright -- I'll start work on getting the script which combines them into a single .csv file today/tomorrow.

River-segment scale EVAP stored in .wdms in /opt/model/p6/p6_gb604/input/scenario/climate/met/N20150521J96 naming convention: met_DE0_5650_0000.wdm

River-segment scale PRCP stored in .wdms in /opt/model/p6/p6_gb604/input/scenario/climate/prad/PRC20170731 naming convention: prad_DE0_5650_0000.wdm

For future reference, where are these UCI files stored? They seem to link all the model components together, but I don't think I've ever seen or dug into one.

[hdaniel7]

Progress as of yet:

EVAP .csv files for all land and river segments has been generated to the new directory http://deq2.bse.vt.edu/p6/p6_gb604/out/climate/ using the command wdm.to.csv.all(wdmpath = '/opt/model/p6/p6_gb604/input/scenario/climate/met/N20150521J96', outputdir = '/opt/model/p6/p6_gb604/out/climate', 1984, 2014, 1000) [from the openmi-om repository]

PRCP .csv files for all land and river segments has also been generated to the new directory http://deq2.bse.vt.edu/p6/p6_gb604/out/climate/ using the command wdm.to.csv.all(wdmpath = '/opt/model/p6/p6_gb604/input/scenario/climate/prad/PRC20170731', outputdir = '/opt/model/p6/p6_gb604/out/climate', 1984, 2014, 2000)

As a test, successfully created file "DE0_5650_0000_1000-2000.csv" at "http://deq2.bse.vt.edu/p6/p6_gb604/out/climate/" using a new function stored in the file "openmi-om/R/utils/fn.climate_evap.and.prcp" -- as part of this process, the separate EVAP and PRCP files that had been generated in the last two steps were deleted to save space. Syntax for the new function was: climate_evap.and.prcp('DE0_5650_0000', '/opt/model/p6/p6_gb604', '/opt/model/p6/p6_gb604/out/climate')

I also threw together a quick function "batch.climate" to do the above step for every single land or river segment with evap and prcp files in the given directory -- that is running right now, and all of the .csvs should be generated within a few hours. Syntax for this new function is batch.climate('/opt/model/p6/p6_gb604/out/climate'), and the function file is stored at "/openmi-om/R/utils/batch.climate.R"

Anyways -- check a few of the files in this directory and let me know if that's what you're asking for. Best!

[rburghol]

Fantastic! Thanks, will check them and report back!

[rburghol]

@hdaniel7 this works like a champ. The only think I need help with is which of the coded sub-directories apply to which cliamte scenario -- I see 4 directories, so I just assumed these were the different evap/precip scenario combos. Do you recall this?

5545HS10CA2_and_55R45KK1095/
5545HS50CA2_and_5545KK50AA/ 
5545HS90CA2_and_55R45KK9095/ 
N20150521J96_and_PRC20170731/

[rburghol]

Never mind. I realized how to read in the last couple of minutes!