search

NGEET / fates

repository for the Functionally Assembled Terrestrial Ecosystem Simulator (FATES)
Other
95 stars 91 forks source link

Panama Crane Testbed conversation #297

Closed rosiealice closed 1 year ago

rosiealice commented 6 years ago

We suggested on the 11/7 call that a team effort to getting this testbed up and running might be a way forward. @serbinsh @ckoven @xuchongang @tompowell9 & @rosiealice are plausibly all interested in running the model in Panama, but no-one yet has time to make this their 1st priority. Answering all the questions in this thread might get us some of the way there!

I'm starting this issue as a placeholder for us to talk about coralling the data for the Panama testbed. I'm going to list each of the data streams I think are necessary for running a basic model-data comparison activity with the 'vanilla' version of FATES, along with questions I have about those datasets... I will forget things and I am not in the loop on all of this (RO1 calls conflict with the CLM weekly meeting) so apologies if some of this is a bit basic/repeats stuff you've already said hundreds of times... I will send this as an email as well, and copy replies in here.

Further, @lmkueppers mentioned that this already exists in a master NGEET document from earlier in the project that maybe @charuleka or @jenniferholm has under their jurisdiction. I want to focus this thread on the Panama crane sites but similar efforts could be undertaken for BCI or Manaus etc.

List of data requirements for model testbed:

Boundary Conditions:

Met data: Boris has sent these to Steve Patton, who needs to OK them. @rgknox mentioned he ha scripts to convert to ALM/CLM drivers (and ED2)? Right? This is the most important step in creating the testbed

Soils: Models need one of soil texture and/or water retention curves to properly work. Estimates of soil depth to bedrock and/or the water table are useful too. Joe W or Brent Newman probably the best people to answer this.

Trait Data/Parameters

Leaf level gas change traits: Data exist from @alistairrogers , @walkeranthonyp and @serbinsh field efforts. What other traits did you guys collect? Is there a link to them somewhere we can put here?

Wood density, nutrient traits: Joe has a set of trait data. I forget what it entails except for wood density, LMA & Leaf N.

Root traits: Was there an effort to measure rooting depth? This was discussed early the project.

Validation Data

Sap flow: These are part of RO1 sap flow synthesis and thus exist in reasonable shape. @xuchongang and @bchristo, is that right? Who is the person who is in charge of those files? Are they scaled up (Brad and I have used a script for scaling the Caxiuana data to the canopy scale)

LAI: Do these exist for the crane sites? @xuchongang do you know?

Leaf Litter: Joe had been collecting these?

Tree growth and mortality: Joe had been collecting these?

Soil moisture time series: I recall that these were installed early in the project. Were those data maintained?

Extra stuff for FATES-or ED2-HYDRO

Leaf Water Potential: Brett mentioned that he has predawn and midday LWP data for two years at both crane sites once a month, which is awesome.

Hydraulic traits: Brett also has PV curves (and hydraulic conductivity curves? I didn't hear) @bchristo has his tropical hydraulic trait database.

rgknox commented 6 years ago

@rosiealice, I will format the met-data for FATES compliance.
I have cleaned data from Boris at BCI, updated version.
I do not have any cleaned data for the crane sites. For each site (as they become available), I will create a packet of 1) driver data, 2) surface/domain data and 3) a shell script to run on a generic machine. If improved soils data becomes available, I can help update the surface files. Until I get clarification on the DOI and distribution of policy on the data, I will leave these packets as private access and invite individuals as requested.

serbinsh commented 6 years ago

Great!! Yes Please!

I am at a review panel now so will just give an initial (i.e.quick) response. Indeed we have data from the crane sites, as well as @walkeranthonyp and others including folks at STRI. We have already loaded @walkeranthonyp data into the PEcAn trait database to be used with FATES (and ED, SIPNET, MAAT, etc) but are also working on loading out ENSO and 2017 data into PEcAn as well to use with the modeling efforts. We have already begun work at the tower sites, inclduing testing process representations and eventually testing FATES against our diurnal (leaf-level) measurements, where our gasex and trait data is used for parameterization (e.g. Vcmax, LMA).

Data collected during ENSO and winter 2017: A-Ci, A-Ci by leaf age Diurnal photo-gs, diurnal photo-gs by leaf age leaf water potential (diurnal) leaf chemistry and morphology leaf optical and canopy optical properties seasonal leaf phenology using phenocams (ongoing)

What we have been looking for to push forward the MAAT/FATES (and other model) comparisons with observations: local met drivers for the tower sites, at least up to 2016 but preferably through 2017 (when complete), tree inventory data, soil moisture (available now?), transpiration (available), LAI, canopy temperature, vertical PAR / solar data (i.e. within-canopy) and other relevant comparison data.

We are working on a MAAT evaluation of coupled gs-photo representations to compare with 2016/2017 diurnal data, in a UQ framework but would like to also move this to FATES/ED2 as well.

Of course also we have the results that Jin is writing up in his ENSO paper, that is focused on modeling stomatal slope.

Thus, very interested in working together on this and also providing details on what we are already working on and current data needs.

As for inventory, it would be great to have standardized .css files for ED2/FATES that we can all share for model init conditions. Its nice to test some of the model responses following site-level init and it would be helpful if we had standard inputs to register with PEcAn, etc that all can share....like we have done previously with ED2. THis would indeed make use of @rgknox incorporation of FATES veg init options.

rosiealice commented 6 years ago

This from @alistairrogers on where to find the trait data he uploaded to the data archive:

"...if you go to the archive:

https://ngt-data.lbl.gov/

which requires a username and password

and look at the approved data sets (of which there are 35) and look for data from Ely, Rogers, Serbin, Wolfe and Norby you’ll see all the ENSO Panama data, plus Rich’s LDRD ACi data set. There is still some LANL carbohydrate data in the pipeline that has not made it yet. But to my knowledge that’s all the ENSO data from the BNL/LANL field campaigns. We have some additional 2017 data that is not loaded yet but sampling details are up."

rosiealice commented 6 years ago

This from @sjosephwright "Hi Rosie,

I will provide all data available in Panama. Here, I provide brief answers to your questions.

Boundary Conditions – Stephan Kuepers recently recorded soil water retention curves from BCI. I can get these. Soil depth - Brett Wolfe got down 20 feet at the PNM crane and did not reach bedrock. We often say the soil is 5 m deep on the BCI plateau but that is a guess.

Trait Data – I have attached an incomplete plant traits metadata file. We also have leaf fiber analyses, fracture toughness and elemental composition. Let me know if anyone needs the data. Rooting Depth – Deep. Mel Tyree and I used the construction of the foundation of the BCI lab building to get down to 5 m. There were zero fine roots, but we followed a 1-cm diameter root to 5-m depth, and it was still headed straight down. We have extensive data to 1 or 2 m depth.

Validation data – Brett Wolfe has oodles of sap flow data. LAI – I’ve attached papers by Kaoru Kitajima for individual tree LAI for the PNM crane and by Reiner Wirth for the best stand level estimate for BCI.
Leaf litter – I have leaf litter data from BCI and from both crane sites. Tom Powell already has the BCI data. Tree growth and mortality – There are CTFS plots on BCI and at the San Lorenzo crane. Soil Moisture time series – gravimetric soil moisture content has been determined every two weeks since 1971 on BCI. 1971 is not a typo. There are also two weirs on BCI with many years of flow data.

Extra Stuff – Brett Wolfe is the man for leaf water potentials and hydraulic traits. Jose Medina will have anything that Brett might have missed.

Let me know how to help effectively. This might require (re-) learning Github.

Best,

Joe"

rgknox commented 6 years ago

Update on my tasks (I will be out for two days so I wanted to check in now).

I have a set of scripts (python) that convert the CSV formatted data into the netcdf standard used for FATES. Some amount of evaluation and verification of the data has been done, the scripts are fairly flexible in how the format transformation is performed. I will make these scripts available to the team on github next week (if others want early access drop me a line).

I've started the process of creating a build script that will ingest these driver data and corresponding surface/domain files into a clm-fates simulation. I think I roughly have the main elements of the script in place, and now I'm in the phase of changing units, verifying data, tweeking, etc. If anyone is familiar with the process of reading tower data into CLM, I'm using a flag call CLM_1PT.

In parallel, Boris and I are also re-verifying some of the post-processed data. We have not seen anything weird, but will be taking a look at the data next week, to see if there are any improvements we can add. We will also finalize some attribution text and acknowledgments in the metadata.

@charuleka has provided me with the crane site driver data as well. I now have cleaned csv data from all three sites, but some may not be final versions (thats up to the data team to decide).

I'm hoping that I should have a v0 test-bed package ready by next week for the panama sites. I will work with Charu also to maintain a record of our formatting in these packages (by including the formatting scripts) or other methods of provenance she recommends.

rgknox commented 6 years ago

I created a package for those people listed in Rosie's original message. The package contains a fates-clm build script, met driver data and surface/domain files for BCI.

I'm going to hold on creating a shared link, or other dissemination method, until we get acknowledgements and DOI's squared away. Until that time, the package should be considered "early release" and should not generate publishable research and should NOT be distributed.

tagging: @charuleka

If anyone else wants access, please send me an email.

I conducted some test simulations at the site. Below shows a comparison of a 15 year fates cold-start simulation at BCI using Qian driver data (1990-2004) and the tower's driver data (2008-2014). The numbers look pretty reasonable.

bci_test_v2_plots.pdf

EDIT: Updated date ranges specified in QIan driver above.

rosiealice commented 6 years ago

Hi Ryan I'm totally keen to be a guinea pig for these scripts. Thank you!!

rgknox commented 6 years ago

Rosie, I tested by unpacking it in the cime/scripts directory. Although other locations should work too.

rosiealice commented 6 years ago

Thanks! Will report back (in meeting today though)

On Nov 14, 2017 3:59 PM, "Ryan Knox" notifications@github.com wrote:

Rosie, I tested by unpacking it in the cime/scripts directory. Although other locations should work too.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NGEET/fates/issues/297#issuecomment-344305170, or mute the thread https://github.com/notifications/unsubscribe-auth/AMWsQ3lvoDWFh5Nh-uI42b45QWlo6U33ks5s2bjkgaJpZM4QVcsJ .

lmkueppers commented 6 years ago

Cool! Niggly thing: are the output variables in FATES still called ED_xxxx?  Is that on the to-do list to change?

Less niggly: can this build script start from current stand structure (or a restart file) or just from 0?

Lara

-- Lara M. Kueppers Assistant Professor, Energy & Resources Group, UC Berkeley Faculty Scientist, Climate and Ecosystem Sciences Division, Berkeley Lab 510.486.5813 o. 510.457.6524 c.

ckoven commented 6 years ago

@lmkueppers your Q1: a few of them are: searching through the code reveals ED_NPATCHES, ED_NCOHORTS, ED_bstore, ED_bdead, ED_balive, ED_bleaf, ED_biomass, ED_LITTER_STOCKCOL. some of these make sense to leave as-is (i.e. patches and cohorts are pretty clearly an "ED" concept), the middle chunk we can remove the ED part (and capitalize to follow convention), the last one we could rename FATES_ because a prefix needs to be there to distinguish from HLM litter pool (which exists as well, for complicated reasons).

rgknox commented 6 years ago

@lmkueppers your Q2: Yes this script can be adapted to restart. it can also be adapted to ingest the BCI inventory data as well. For the latter, users need to make it clear that they have signed agreements with ForestGEO and have access to the BCI inventory data, and I can pass them the inventory initialization files and help set the script correctly.

rgknox commented 6 years ago

@serbinsh: "As for inventory, it would be great to have standardized .css files for ED2/FATES that we can all share for model init conditions. " we do have a set of these files generated for FATES/ED2. Have you filed an agreement with the site data providers: http://ctfs.si.edu/webatlas/datasets/bci/. Once that is signed I can pass anyone the css/pss files for the various census years (with a single pft or pioneer/ST pfts).

rgknox commented 6 years ago

Update on Crane Sites:

Updates at BCI:

We have a litany of other parameters to start investigating though, and I think this is the next big step to get started and get more expertise involved again from both the computational field side of things.

I have a parameter list that has some rudimentary rank-ordering here (its also very slightly out of date): https://docs.google.com/spreadsheets/d/13WgBcCzX7GA1A13bzxJwjPFwLI4DSEybBAjSWBmqFUU/edit#gid=0

I'm going to work with Joe to get parameter estimates for some of these.

lmkueppers commented 6 years ago

Ryan - I'm wondering if we should have a conversation with Stuart/BCI folks about making a certain set of inventory data available as model initial conditions in a way that doesn't require you to mediate this request. E.g., have a pointer to the BCI dataserver with the file that you've created for this (in the right format, filtered for understory specialists, or whatever) or have an agreed upon file in the NGEE Tropics archive with metadata stating required acknowledgements.

This wouldn't serve the immediate need Shawn might have, but am thinking about having a standing testbed. Just a thought.

Lara

On 11/21/17 1:30 PM, Ryan Knox wrote:

@serbinsh https://github.com/serbinsh: "As for inventory, it would be great to have standardized .css files for ED2/FATES that we can all share for model init conditions. " we do have a set of these files generated for FATES/ED2. Have you filed an agreement with the site data providers: http://ctfs.si.edu/webatlas/datasets/bci/. Once that is signed I can pass anyone the css/pss files for the various census years (with a single pft or pioneer/ST pfts).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NGEET/fates/issues/297#issuecomment-346167338, or mute the thread https://github.com/notifications/unsubscribe-auth/ASpkpX-UV-n-vKEMHMU5KzpSc9FuackYks5s40DygaJpZM4QVcsJ.

-- Lara M. Kueppers Assistant Professor, Energy & Resources Group, UC Berkeley Faculty Scientist, Climate and Ecosystem Sciences Division, Berkeley Lab 510.486.5813 o. 510.457.6524 c.

rgknox commented 6 years ago

@lmkueppers, that would be helpful. Of relevance, the fates formatted files will likely go through various version updates and variants depending on how we classify plants into PFTs. The current files being offered are a single tropical pft.

rosiealice commented 6 years ago

Update from Boris:

Hi All:

The following link https://drive.google.com/open?id=1bDSNJYf1vybH78JtEF9f6kHJwua2xcFE provides an access to the Google Drive folder with the latest QA/QC-ed csv files of the San Lorenzo/Fort Sherman and PNM sites along with a readme (summary) files. Note that the processed files were uploaded by Steve Paton at the http://stri.si.edu/ website on December 11, 2017.

A description of the QA/QC procedure and flagging is given in the Word file.

Please let me know if you have any questions or concerns.

Happy Holidays!

Boris

--

rosiealice commented 6 years ago

Does anyone have a script to covert these the FATES driver netcdf files?

mdietze commented 6 years ago

Haven't given it a try, but PEcAn's met2CF.csv should be able to convert those files to netCDF CF, and then met2model.FATES should convert that to FATE's units and variable names. There's also a met2model.ED2 if you want to convert to ED2's hdf5 met format (as well as functions for all the other models we support)

See "30.1.5 Processing Met data outside of the workflow using PEcAn functions" in the PEcAn documentation here: https://pecanproject.github.io/pecan-documentation/develop/input-conversion.html

rgknox commented 6 years ago

I can convert the met data @rosiealice

serbinsh commented 6 years ago

@rgknox I filled out the STRI form for access to the BCI 50 ha plot data, could you share with me the css/pss files for FATES? Also, I seem to have lost the link/files for running FATES using the BCI driver data, etc. Could you send that to me again or is there another more permanent place I can grab the drivers and build scripts, etc?

I seem to have lost track of the previous email chain. Thanks!

rgknox commented 6 years ago

will do @serbinsh, stay tuned

charuleka commented 6 years ago

Hi all,

Please upload all datasets/scripts that are being used in the NGT archive instead of sharing via other means. For citation, we have language to acknowledge STRI, as used in Tom's published dataset for the BCI drivers.

Thanks Charu.

On Tue, Jan 2, 2018 at 1:38 PM, Ryan Knox notifications@github.com wrote:

will do @serbinsh https://github.com/serbinsh, stay tuned

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NGEET/fates/issues/297#issuecomment-354881857, or mute the thread https://github.com/notifications/unsubscribe-auth/ALT8FFsI8U0mkdQMmWJ90H0BDsABv_qhks5tGqHHgaJpZM4QVcsJ .

serbinsh commented 6 years ago

@charuleka I understand eventually permanently archiving scripts once they are stable but while there are in development I would think its easier to share via git/google?

I say that as they may keep evolving and it may be better to allow them to change without having to modify the archive until we have a "stable release version"

charuleka commented 6 years ago

Yes. FYI You can also point to git repositories from the NGEET archive, instead of uploading the files again there. It seems like the NGEET git has a bunch of repos for various modeling tools - a couple of them haven't been used in a while, but it might be a place to put scripts in development.

On Tue, Jan 2, 2018 at 1:44 PM, Shawn P. Serbin notifications@github.com wrote:

@charuleka https://github.com/charuleka I understand eventually permanently archiving scripts once they are stable but while there are in development I would think its easier to share via git/google?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NGEET/fates/issues/297#issuecomment-354883084, or mute the thread https://github.com/notifications/unsubscribe-auth/ALT8FG61C7c3a6JqwTjaLOmPQJfXNilTks5tGqM5gaJpZM4QVcsJ .

serbinsh commented 6 years ago

@rgknox Not sure if this is the place to post this question but I ran into an issue trying to run a test BCI simulation with the v0 tower drivers:

[sserbin@modex test0.ICLM45ED.modex.gnu.Cb753898-F1ea1cc4.2018-01-02]$ ./check_input_data
Loading input file list: 'Buildconf/datm.input_data_list'
Loading input file list: 'Buildconf/clm.input_data_list'
Model clm no file specified for finidat
  Model clm missing file fates_inventory_ctrl_filename = '/data/sserbin/Modeling/fates-clm/cime/scripts/bci_inv_file_list.txt'
  Model clm missing file fates_paramfile = '/data/sserbin/Modeling/fates-clm/cime/scripts/bci_met.v1_surf.v0/ParameterFiles/fates_params_1tropbci_SOdef.c171121.nc'
Loading input file list: 'Buildconf/rtm.input_data_list'
Loading input file list: 'Buildconf/cpl.input_data_list'
[sserbin@modex test0.ICLM45ED.modex.gnu.Cb753898-F1ea1cc4.2018-01-02]$ ./case.submit
Creating component namelists
   Calling datm buildnml
   Calling clm buildnml
   Calling sice buildnml
   Calling socn buildnml
   Calling rtm buildnml
   Calling sglc buildnml
   Calling swav buildnml
   Calling sesp buildnml
   Calling drv buildnml
Finished creating component namelists
Checking that inputdata is available as part of case submission
Loading input file list: 'Buildconf/datm.input_data_list'
Loading input file list: 'Buildconf/clm.input_data_list'
  Model clm missing file fates_inventory_ctrl_filename = '/data/sserbin/Modeling/fates-clm/cime/scripts/bci_inv_file_list.txt'
    Cannot download file since it lives outside of the input_data_root '/data/Model_Data/cesm_input_datasets/'
Model clm no file specified for finidat
  Model clm missing file fates_paramfile = '/data/sserbin/Modeling/fates-clm/cime/scripts/bci_met.v1_surf.v0/ParameterFiles/fates_params_1tropbci_SOdef.c171121.nc'
    Cannot download file since it lives outside of the input_data_root '/data/Model_Data/cesm_input_datasets/'
Loading input file list: 'Buildconf/rtm.input_data_list'
Loading input file list: 'Buildconf/cpl.input_data_list'
ERROR: Failed to download input data
[sserbin@modex test0.ICLM45ED.modex.gnu.Cb753898-F1ea1cc4.2018-01-02]$

I put the drivers and scripts, as recommended in the header of create_bci_panama.sh, into cime/scripts and edited as:


export SITE_NAME=bci_met.v1_surf.v0                        # Name of folder with site data
export SITE_BASE_DIR=`pwd`                                 # Where is the site folder located? (SITE_NAME)
export TAG='test0'                                         # User defined tag to differentiate runs
export COMPSET=ICLM45ED                                    # Compset (probably ICLM45ED or ICLM50ED)
export MAC=modex                                           # Name your machine
export COMPILER=gnu                                        # Name your compiler
export CASEROOT=/data/Model_Output/fates_testing/BCI/      # Where the build is generated (probably on scratch partition)
export CLM_USRDAT_DOMAIN=domain_bci_panama_v1_c171113.nc   # Name of domain file in scripts/${SITE_DIR}/
export CLM_USRDAT_SURDAT=surfdata_bci_panama_v1_c171113.nc # Name of surface file in scripts/${SITE_DIR}/

This is because my modex machine files have a different default cesm_inputs dir. Should I just edit these lines to reflect?

fates_paramfile = '${SITE_BASE_DIR}/${SITE_NAME}/ParameterFiles/fates_params_1tropbci_SOdef.c171121.nc'
fates_inventory_ctrl_filename = '${SITE_BASE_DIR}/bci_inv_file_list.txt'
rgknox commented 6 years ago

thanks for product testing @serbinsh , I need to fix that package. You can just remove those two entries for fates_paramfile and fates_inventory_ctrl_filename.

serbinsh commented 6 years ago

Oh ok. Yeah I see that even if I change the path to my cesm_inputs dir it fails because it can't download those files....

rgknox commented 6 years ago

also remove the line: use_fates_inventory_init = .true. see: https://github.com/NGEET/fates/wiki/Model-Initialization-Modes#Guide_for_a_Cold_Start_Simulation_with_Inventory_Case_3

rgknox commented 6 years ago

Update: I'm working on processing PNM and Fort Sherman. PNM does not have wind-speed data. I'm currently looking into ways of grabbing that from a different dataset, but I think this deserves a conversation, as we would be doing bad things like using data not associated with the site, and breaking correlation of variables.

rosiealice commented 6 years ago

I messaged Boris to ask about wind speed data... I bet there is some somewhere.

serbinsh commented 6 years ago

@rgknox Update: removing those flags fixed my issues with the BCI test run.

charuleka commented 6 years ago

Boris' reply

---------- Forwarded message ---------- From: Boris Faybishenko bafaybishenko@lbl.gov Date: Wed, Jan 3, 2018 at 7:03 AM Subject: Re: [NGEET/fates] Panama Crane Testbed conversation (#297) To: Ryan Knox rgknox@lbl.gov Cc: Charuleka Varadharajan cvaradharajan@lbl.gov

HI Ryan:

I processed Fort Sherman wind-speed and wind direction, and included them in the QA/QC-ed dataset that I sent before the Holidays.

Regarding the PNM site: it is not currently available online, and I will contact Steve Paton when he is back in the office on January 14. I agree that taking these data from another site may break correlation between the variables.

Thank you, Boris

On Tue, Jan 2, 2018 at 3:22 PM, Charuleka Varadharajan < cvaradharajan@lbl.gov> wrote:

FYI ---------- Forwarded message ---------- From: Ryan Knox notifications@github.com Date: Tue, Jan 2, 2018 at 3:21 PM Subject: Re: [NGEET/fates] Panama Crane Testbed conversation (#297) To: NGEET/fates fates@noreply.github.com Cc: charuleka cvaradharajan@lbl.gov, Mention <mention@noreply.github.com

Update: I'm working on processing PNM and Fort Sherman. PNM does not have wind-speed data. I'm currently looking into ways of grabbing that from a different dataset, but I think this deserves a conversation, as we would be doing bad things like using data not associated with the site, and breaking correlation of variables.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NGEET/fates/issues/297#issuecomment-354901959, or mute the thread https://github.com/notifications/unsubscribe-auth/ALT8FGfGjxr7E3V7zFl_l62_jGr2g3c4ks5tGroVgaJpZM4QVcsJ .

-- Boris Faybishenko Earth and Environmental Sciences Area Energy Geosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720 510-486-4852 <(510)%20486-4852> bafaybishenko@lbl.gov

On Tue, Jan 2, 2018 at 3:26 PM, rosiealice notifications@github.com wrote:

I messaged Boris to ask about wind speed data... I bet there is some somewhere.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NGEET/fates/issues/297#issuecomment-354902597, or mute the thread https://github.com/notifications/unsubscribe-auth/ALT8FN7Oor_SGY_B2mtlgaC8-X4hxd_oks5tGrsKgaJpZM4QVcsJ .

rgknox commented 6 years ago

@rosiealice and co:

"Soils: Models need one of soil texture and/or water retention curves to properly work. Estimates of soil depth to bedrock and/or the water table are useful too. Joe W or Brent Newman probably the best people to answer this."

I'm currently setting up the surface data at Sherman. As a first pass, I can just use the nearest neighbor on: https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/lnd/clm2/surfdata_map/surfdata_1.9x2.5_16pfts_simyr2000_c160127.nc

But, if better data has been wrangled already, I can get it in there now.

Has anyone rounded up the site level soils data for Sherman, PMN or BCI yet?

rgknox commented 6 years ago

Here is an ncdump on the surface dataset. I have not gone through and determined which of these are immediately relevant in FATES though:

variables:
    int mxsoil_color ;
        mxsoil_color:long_name = "maximum numbers of soil colors" ;
        mxsoil_color:units = "unitless" ;
    int SOIL_COLOR(lsmlat, lsmlon) ;
        SOIL_COLOR:long_name = "soil color" ;
        SOIL_COLOR:units = "unitless" ;
    double PCT_SAND(nlevsoi, lsmlat, lsmlon) ;
        PCT_SAND:long_name = "percent sand" ;
        PCT_SAND:units = "unitless" ;
    double PCT_CLAY(nlevsoi, lsmlat, lsmlon) ;
        PCT_CLAY:long_name = "percent clay" ;
        PCT_CLAY:units = "unitless" ;
    double ORGANIC(nlevsoi, lsmlat, lsmlon) ;
        ORGANIC:long_name = "organic matter density at soil levels" ;
        ORGANIC:units = "kg/m3 (assumed carbon content 0.58 gC per gOM)" ;
    double FMAX(lsmlat, lsmlon) ;
        FMAX:long_name = "maximum fractional saturated area" ;
        FMAX:units = "unitless" ;
    int natpft(natpft) ;
        natpft:long_name = "indices of natural PFTs" ;
        natpft:units = "index" ;
    double LANDFRAC_PFT(lsmlat, lsmlon) ;
        LANDFRAC_PFT:long_name = "land fraction from pft dataset" ;
        LANDFRAC_PFT:units = "unitless" ;
    int PFTDATA_MASK(lsmlat, lsmlon) ;
        PFTDATA_MASK:long_name = "land mask from pft dataset, indicative of real/fake points" ;
        PFTDATA_MASK:units = "unitless" ;
    double PCT_NATVEG(lsmlat, lsmlon) ;
        PCT_NATVEG:long_name = "total percent natural vegetation landunit" ;
        PCT_NATVEG:units = "unitless" ;
    double PCT_CROP(lsmlat, lsmlon) ;
        PCT_CROP:long_name = "total percent crop landunit" ;
        PCT_CROP:units = "unitless" ;
    double PCT_NAT_PFT(natpft, lsmlat, lsmlon) ;
        PCT_NAT_PFT:long_name = "percent plant functional type on the natural veg landunit (% of landunit)" ;
        PCT_NAT_PFT:units = "unitless" ;
    double MONTHLY_LAI(time, lsmpft, lsmlat, lsmlon) ;
        MONTHLY_LAI:long_name = "monthly leaf area index" ;
        MONTHLY_LAI:units = "unitless" ;
    double MONTHLY_SAI(time, lsmpft, lsmlat, lsmlon) ;
        MONTHLY_SAI:long_name = "monthly stem area index" ;
        MONTHLY_SAI:units = "unitless" ;
    double MONTHLY_HEIGHT_TOP(time, lsmpft, lsmlat, lsmlon) ;
        MONTHLY_HEIGHT_TOP:long_name = "monthly height top" ;
        MONTHLY_HEIGHT_TOP:units = "meters" ;
    double MONTHLY_HEIGHT_BOT(time, lsmpft, lsmlat, lsmlon) ;
        MONTHLY_HEIGHT_BOT:long_name = "monthly height bottom" ;
        MONTHLY_HEIGHT_BOT:units = "meters" ;
    int time(time) ;
        time:long_name = "Calendar month" ;
        time:units = "month" ;
    double AREA(lsmlat, lsmlon) ;
        AREA:long_name = "area" ;
        AREA:units = "km^2" ;
    double LONGXY(lsmlat, lsmlon) ;
        LONGXY:long_name = "longitude" ;
        LONGXY:units = "degrees east" ;
    double LATIXY(lsmlat, lsmlon) ;
        LATIXY:long_name = "latitude" ;
        LATIXY:units = "degrees north" ;
    double EF1_BTR(lsmlat, lsmlon) ;
        EF1_BTR:long_name = "EF btr (isoprene)" ;
        EF1_BTR:units = "unitless" ;
    double EF1_FET(lsmlat, lsmlon) ;
        EF1_FET:long_name = "EF fet (isoprene)" ;
        EF1_FET:units = "unitless" ;
    double EF1_FDT(lsmlat, lsmlon) ;
        EF1_FDT:long_name = "EF fdt (isoprene)" ;
        EF1_FDT:units = "unitless" ;
    double EF1_SHR(lsmlat, lsmlon) ;
        EF1_SHR:long_name = "EF shr (isoprene)" ;
        EF1_SHR:units = "unitless" ;
    double EF1_GRS(lsmlat, lsmlon) ;
        EF1_GRS:long_name = "EF grs (isoprene)" ;
        EF1_GRS:units = "unitless" ;
    double EF1_CRP(lsmlat, lsmlon) ;
        EF1_CRP:long_name = "EF crp (isoprene)" ;
        EF1_CRP:units = "unitless" ;
    double CANYON_HWR(numurbl, lsmlat, lsmlon) ;
        CANYON_HWR:long_name = "canyon height to width ratio" ;
        CANYON_HWR:units = "unitless" ;
    double EM_IMPROAD(numurbl, lsmlat, lsmlon) ;
        EM_IMPROAD:long_name = "emissivity of impervious road" ;
        EM_IMPROAD:units = "unitless" ;
    double EM_PERROAD(numurbl, lsmlat, lsmlon) ;
        EM_PERROAD:long_name = "emissivity of pervious road" ;
        EM_PERROAD:units = "unitless" ;
    double EM_ROOF(numurbl, lsmlat, lsmlon) ;
        EM_ROOF:long_name = "emissivity of roof" ;
        EM_ROOF:units = "unitless" ;
    double EM_WALL(numurbl, lsmlat, lsmlon) ;
        EM_WALL:long_name = "emissivity of wall" ;
        EM_WALL:units = "unitless" ;
    double HT_ROOF(numurbl, lsmlat, lsmlon) ;
        HT_ROOF:long_name = "height of roof" ;
        HT_ROOF:units = "meters" ;
    double THICK_ROOF(numurbl, lsmlat, lsmlon) ;
        THICK_ROOF:long_name = "thickness of roof" ;
        THICK_ROOF:units = "meters" ;
    double THICK_WALL(numurbl, lsmlat, lsmlon) ;
        THICK_WALL:long_name = "thickness of wall" ;
        THICK_WALL:units = "meters" ;
    double T_BUILDING_MAX(numurbl, lsmlat, lsmlon) ;
        T_BUILDING_MAX:long_name = "maximum interior building temperature" ;
        T_BUILDING_MAX:units = "K" ;
    double T_BUILDING_MIN(numurbl, lsmlat, lsmlon) ;
        T_BUILDING_MIN:long_name = "minimum interior building temperature" ;
        T_BUILDING_MIN:units = "K" ;
    double WIND_HGT_CANYON(numurbl, lsmlat, lsmlon) ;
        WIND_HGT_CANYON:long_name = "height of wind in canyon" ;
        WIND_HGT_CANYON:units = "meters" ;
    double WTLUNIT_ROOF(numurbl, lsmlat, lsmlon) ;
        WTLUNIT_ROOF:long_name = "fraction of roof" ;
        WTLUNIT_ROOF:units = "unitless" ;
    double WTROAD_PERV(numurbl, lsmlat, lsmlon) ;
        WTROAD_PERV:long_name = "fraction of pervious road" ;
        WTROAD_PERV:units = "unitless" ;
    double ALB_IMPROAD_DIR(numrad, numurbl, lsmlat, lsmlon) ;
        ALB_IMPROAD_DIR:long_name = "direct albedo of impervious road" ;
        ALB_IMPROAD_DIR:units = "unitless" ;
    double ALB_IMPROAD_DIF(numrad, numurbl, lsmlat, lsmlon) ;
        ALB_IMPROAD_DIF:long_name = "diffuse albedo of impervious road" ;
        ALB_IMPROAD_DIF:units = "unitless" ;
    double ALB_PERROAD_DIR(numrad, numurbl, lsmlat, lsmlon) ;
        ALB_PERROAD_DIR:long_name = "direct albedo of pervious road" ;
        ALB_PERROAD_DIR:units = "unitless" ;
    double ALB_PERROAD_DIF(numrad, numurbl, lsmlat, lsmlon) ;
        ALB_PERROAD_DIF:long_name = "diffuse albedo of pervious road" ;
        ALB_PERROAD_DIF:units = "unitless" ;
    double ALB_ROOF_DIR(numrad, numurbl, lsmlat, lsmlon) ;
        ALB_ROOF_DIR:long_name = "direct albedo of roof" ;
        ALB_ROOF_DIR:units = "unitless" ;
    double ALB_ROOF_DIF(numrad, numurbl, lsmlat, lsmlon) ;
        ALB_ROOF_DIF:long_name = "diffuse albedo of roof" ;
        ALB_ROOF_DIF:units = "unitless" ;
    double ALB_WALL_DIR(numrad, numurbl, lsmlat, lsmlon) ;
        ALB_WALL_DIR:long_name = "direct albedo of wall" ;
        ALB_WALL_DIR:units = "unitless" ;
    double ALB_WALL_DIF(numrad, numurbl, lsmlat, lsmlon) ;
        ALB_WALL_DIF:long_name = "diffuse albedo of wall" ;
        ALB_WALL_DIF:units = "unitless" ;
    double TK_ROOF(nlevurb, numurbl, lsmlat, lsmlon) ;
        TK_ROOF:long_name = "thermal conductivity of roof" ;
        TK_ROOF:units = "W/m*K" ;
    double TK_WALL(nlevurb, numurbl, lsmlat, lsmlon) ;
        TK_WALL:long_name = "thermal conductivity of wall" ;
        TK_WALL:units = "W/m*K" ;
    double TK_IMPROAD(nlevurb, numurbl, lsmlat, lsmlon) ;
        TK_IMPROAD:long_name = "thermal conductivity of impervious road" ;
        TK_IMPROAD:units = "W/m*K" ;
    double CV_ROOF(nlevurb, numurbl, lsmlat, lsmlon) ;
        CV_ROOF:long_name = "volumetric heat capacity of roof" ;
        CV_ROOF:units = "J/m^3*K" ;
    double CV_WALL(nlevurb, numurbl, lsmlat, lsmlon) ;
        CV_WALL:long_name = "volumetric heat capacity of wall" ;
        CV_WALL:units = "J/m^3*K" ;
    double CV_IMPROAD(nlevurb, numurbl, lsmlat, lsmlon) ;
        CV_IMPROAD:long_name = "volumetric heat capacity of impervious road" ;
        CV_IMPROAD:units = "J/m^3*K" ;
    int NLEV_IMPROAD(numurbl, lsmlat, lsmlon) ;
        NLEV_IMPROAD:long_name = "number of impervious road layers" ;
        NLEV_IMPROAD:units = "unitless" ;
    double peatf(lsmlat, lsmlon) ;
        peatf:long_name = "peatland fraction" ;
        peatf:units = "unitless" ;
    double zbedrock(lsmlat, lsmlon) ;
        zbedrock:long_name = "soil depth" ;
        zbedrock:units = "m" ;
    int abm(lsmlat, lsmlon) ;
        abm:long_name = "agricultural fire peak month" ;
        abm:units = "unitless" ;
    double gdp(lsmlat, lsmlon) ;
        gdp:long_name = "gdp" ;
        gdp:units = "unitless" ;
    double SLOPE(lsmlat, lsmlon) ;
        SLOPE:long_name = "mean topographic slope" ;
        SLOPE:units = "degrees" ;
    double STD_ELEV(lsmlat, lsmlon) ;
        STD_ELEV:long_name = "standard deviation of elevation" ;
        STD_ELEV:units = "m" ;
    double binfl(lsmlat, lsmlon) ;
        binfl:long_name = "VIC b parameter for the Variable Infiltration Capacity Curve" ;
        binfl:units = "unitless" ;
    double Ws(lsmlat, lsmlon) ;
        Ws:long_name = "VIC Ws parameter for the ARNO curve" ;
        Ws:units = "unitless" ;
    double Dsmax(lsmlat, lsmlon) ;
        Dsmax:long_name = "VIC Dsmax parameter for the ARNO curve" ;
        Dsmax:units = "mm/day" ;
    double Ds(lsmlat, lsmlon) ;
        Ds:long_name = "VIC Ds parameter for the ARNO curve" ;
        Ds:units = "unitless" ;
    double LAKEDEPTH(lsmlat, lsmlon) ;
        LAKEDEPTH:long_name = "lake depth" ;
        LAKEDEPTH:units = "m" ;
    double F0(lsmlat, lsmlon) ;
        F0:long_name = "maximum gridcell fractional inundated area" ;
        F0:units = "unitless" ;
    double P3(lsmlat, lsmlon) ;
        P3:long_name = "coefficient for qflx_surf_lag for finundated" ;
        P3:units = "s/mm" ;
    double ZWT0(lsmlat, lsmlon) ;
        ZWT0:long_name = "decay factor for finundated" ;
        ZWT0:units = "m" ;
    double PCT_WETLAND(lsmlat, lsmlon) ;
        PCT_WETLAND:long_name = "percent wetland" ;
        PCT_WETLAND:units = "unitless" ;
    double PCT_LAKE(lsmlat, lsmlon) ;
        PCT_LAKE:long_name = "percent lake" ;
        PCT_LAKE:units = "unitless" ;
    double PCT_GLACIER(lsmlat, lsmlon) ;
        PCT_GLACIER:long_name = "percent glacier" ;
        PCT_GLACIER:units = "unitless" ;
    int GLACIER_REGION(lsmlat, lsmlon) ;
        GLACIER_REGION:long_name = "glacier region ID" ;
        GLACIER_REGION:units = "unitless" ;
    double GLC_MEC(nglcecp1) ;
        GLC_MEC:long_name = "Glacier elevation class" ;
        GLC_MEC:units = "m" ;
    double PCT_GLC_MEC(nglcec, lsmlat, lsmlon) ;
        PCT_GLC_MEC:long_name = "percent glacier for each glacier elevation class (% of landunit)" ;
        PCT_GLC_MEC:units = "unitless" ;
    double PCT_GLC_MEC_GIC(nglcec, lsmlat, lsmlon) ;
        PCT_GLC_MEC_GIC:long_name = "percent smaller glaciers and ice caps for each glacier elevation class (% of landunit)" ;
        PCT_GLC_MEC_GIC:units = "unitless" ;
    double PCT_GLC_MEC_ICESHEET(nglcec, lsmlat, lsmlon) ;
        PCT_GLC_MEC_ICESHEET:long_name = "percent ice sheet for each glacier elevation class (% of landunit)" ;
        PCT_GLC_MEC_ICESHEET:units = "unitless" ;
    double PCT_GLC_GIC(lsmlat, lsmlon) ;
        PCT_GLC_GIC:long_name = "percent ice caps/glaciers (% of landunit)" ;
        PCT_GLC_GIC:units = "unitless" ;
    double PCT_GLC_ICESHEET(lsmlat, lsmlon) ;
        PCT_GLC_ICESHEET:long_name = "percent ice sheet (% of landunit)" ;
        PCT_GLC_ICESHEET:units = "unitless" ;
    double TOPO_GLC_MEC(nglcec, lsmlat, lsmlon) ;
        TOPO_GLC_MEC:long_name = "mean elevation on glacier elevation classes" ;
        TOPO_GLC_MEC:units = "m" ;
    double PCT_URBAN(numurbl, lsmlat, lsmlon) ;
        PCT_URBAN:long_name = "percent urban for each density type" ;
        PCT_URBAN:units = "unitless" ;
    int URBAN_REGION_ID(lsmlat, lsmlon) ;
        URBAN_REGION_ID:long_name = "urban region ID" ;
        URBAN_REGION_ID:units = "unitless" ;
tompowell9 commented 6 years ago

We have soil texture and bulk density to 2-4m depth at BCI. These data are for the plateau, well-drained slope and poorly-drained slope.

alistairrogers commented 6 years ago

David (working with Joe and Brett) was collecting soil moisture content data at SLZ and PNM during the ENSO campaign so Brett should have access to that data and I believe they were also going to do rention curves. Ben Turner would be an obvious person to ask too. A

Sent from my iPhone

On Jan 5, 2018, at 5:57 PM, tompowell9 notifications@github.com<mailto:notifications@github.com> wrote:

We have soil texture and bulk density to 2-4m depth at BCI. These data are for the plateau, well-drained slope and poorly-drained slope.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/NGEET/fates/issues/297#issuecomment-355688283, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ANTfrSm4_TGFP2TyBqjAiEI2xppCrL9pks5tHqjogaJpZM4QVcsJ.

rosiealice commented 1 year ago

This is ancient now....