Add CLM

[mdietze]

Had a chat with Dan Ricciutto at NACP2015 about the specifics of doing CLM4.5 site level runs and coupling to PEcAn and this definitely looks do-able for the initial task of doing runs where the drivers are already in the database. The harder challenge will be spin up support #347.

The first task is that Dan will download a copy of the PEcAn VM and install CLM4.5 and his site run scripts there and all dependencies. He will make notes of the steps involved in doing this. He will then send me the VM, his notes, and the scripts he uses to generate MsTMIP output files. @robkooper will want to check that he can install CLM onto a clean VM based on these notes.

I will then take a first pass at setting up the BETYdb records for CLM (including all the Input records required for a single site run) and writing write.configs.CLM45 and model2netcdf.CLM45.

After that, we'll work on met2CF.CLM45

[mdietze]

I've set up the most basic BETY records for CLM4.5 and confirmed that it can be installed on the VM. @tonygardella needs to repeat this independently, confirm that we can do a site-level test run for one of Dan's default sites, and then start working on how to replicate that run within PEcAn by writing write.configs.CLM45 and model2netcdf.CLM45. The first run should not accept any parameters and should just run for that one test site (met input should be logged with the database, for now we can assume all other inputs are defaults) and then convert the output to standard format. For the output conversion, please check in with Dan R since he undoubtedly has scripts for that already (since the output standard is the same as NACP site synthesis and MsTMIP).

[tonygardella]

• [x] get a demo run to complete on VM
• [ ] get a reference case to complete on BU servers #714
• [ ] get that same case to complete through write.configs
• [ ] model2netcdf: get that case to give output PEcAn can read
• [ ] met2model: generalize met to CLM requirements
• [ ] modify write.configs to allow lat, lon, start date, end date, and met to be controlled by pecan
• [ ] add additional required inputs to the model type table and load those defaults into Inputs/dbfiles table
• [ ] create a test PFT with 2-3 model parameters and priors
• [ ] modify write.configs to allow those parameters to be controlled by pecan. More generally, map parameters from BETY variables and expand PFTs & write config for more variables
• [ ] add support for “restart” files (write.restart and read.restart functions)
• [ ] add PEcAn processing of other required inputs and optional inputs
• [ ] add PEcAn processing of other optional inputs

[mdietze]

We had a discussion on the CLM-ED coupling at the ED+ meeting at NCAR last week (myself, @bcow, @Viskari, Rosie Fisher, Ben Sanderson, @mvertens, @rgknox) and came up for a plan on how to leverage CIME https://github.com/ESMCI/cime to do the coupling to CLM, CLM-ED, and ACME. Below are the contents of an email I wrote last night describing the plan

I had a chance to chat with Tony on Monday to formulate a plan and then we shared some of this with Shawn and others during the weekly PEcAn telecon today. We envisioned the coupling proceeding in a number of stages (similar to what we’ve done with other models), with each stage adding functionality — apologies in advance, the somewhat odd numbering below corresponds to the picture of my whiteboard following my meeting with Tony and I’ve not renumbered just to stay consistent. The plan below is also largely consistent with our previous plan, https://github.com/PecanProject/pecan/issues/352 , but with a lot more detail filled in

Step 1 is just to get CLM compiled and a reference case running on both the BU and BNL PEcAn servers. By working on both machines simultaneously we’re hoping we get this figured out in general, not just as a special case. Earlier, Dan Ricciuto helped Tony get the CLM4.5 public release running on the PEcAn VM but it turns out Tony’s not yet able to replicate that on our own server (https://github.com/PecanProject/pecan/issues/714) and Dan’s not had as much time to help out. I’m hoping you and Ben will be able to get us through this -- expect questions fairly soon!

Current progress:

• I started downloading the development branch yesterday — admittedly I’ve had the occasional interrupt and manual restart, but it’s over a day latter and I’m still downloading. This thing is huge!
• Tony is going to type up the proposed plan (below) as pseudocode directly into the write.config.CLM function. He’ll then do a pull request so the whole team can see and discuss that plan. We’re hoping that Shawn can tackle a few subtasks and would love help from others as well.

Step 2: Get PEcAn coupler to run the reference case (i.e. with write.configs ignoring all inputs from PEcAn) A) within write.configs, copy the reference CASE and BUILD, setting up symbolic links to executable B) have write.configs write PEcAn’s jobs.sh script which would * ./case.setup * ./case.submit * call the PEcAn script to convert CLM output to PEcAn standard and put it in the output directory where PEcAn’s expecting to find it.

Step 3: Have PEcAn run a very basic forward run for a new location C) point met to PEcAn’s input within user_nl_datm D) set lat/lon in a 1x1 grid in a domain file E) set start date, end data, and domain file location in env_run.xml

Step 4: Give PEcAn control of a few parameters F) have PEcAn write a CLM netcdf parameter file and point to it in user_nl_clm Ben and Rosie are responsible for convincing the CLM community to stop putting parameters in other places. To provide them with fodder for doing so, I unilaterally decided that PEcAn is not going to try and keep track of parameters in other odd locations but will only support the netCDF parameter file. F’ ) Once PEcAn can successfully support a few parameters, I’m hoping the rest of you can use that as a template and help with the relatively simple but time consuming task of adding more variables, priors, and PFTs

Step 5: Give PEcAn control of Initial Conditions G1) Start with PEcAn supporting bare ground runs G2) Run a global spin-up as a reference case G3) @ runtime, have PEcAn extract the closest grid cell from the reference run and use as the IC for the point. This will be written as a case.clm.h0.[date].nc file and referenced as finidat in user_nl_clm G4) Write PEcAn’s read.restart function for CLM G5) Write PEcAn’s write.restart function for CLM G6) use G4 and G5 to have PEcAn control veg IC within h0 file. This intersects with other work in the project to generalize our current IC support beyond ED2 (e.g. FIA veg). At the moment PEcAn isn’t setting soil C IC for any model, but that’s part of the plan A useful side effect of G4 & G5 is that these are also the functions we need to get CLM working in our state data assimilation code as well

Step 6: Give PEcAn control of soils and other edaphic variables H1) write these variables into surfdat.[stuff].nc file H2) set a pointer to this file in user_nl_clm under fsurdat As with soil C, PEcAn also isn’t setting soil texture, etc automatically for any model but that’s likewise part of the plan

Beyond this, we can gradually give PEcAn control over more inputs (land use, N dep, etc) using the same pattern, but I suspect what’s described above will cover the needs of >99% of all users and everyone else will be happy with CLM’s defaults.

[dlebauer]

@pengbinpeluo FYI

[mdietze]

CLM repository has now shifted to a public Github repo https://github.com/ESCOMP/ctsm

@tonygardella you might want to try forking, cloning to our servers, and building. There seems to be a separate documentation repo as well. Hopefully this move will allow us to open issues when we encounter problems building.

[tonygardella]

@serbinsh Have you been able to compile and run CLM5 from the CTSM github repository?

[dlebauer]

Assigning to @amfox37 ...

@robkooper or @tonygardella can you confirm that you got CLM running on the VM as stated in #714 ? Do you have notes on this?

[tonygardella]

@dlebauer Initial trial that resulted in it working on the VM: runscript: https://gist.github.com/tonygardella/44b79a43989bf346c0c82a854b7d9b0e Instructions to setup and use script: https://gist.github.com/tonygardella/d5458bf75119cbf5002a722161796ca6

That being said this for old CLM 4.5 and the moment this was tried on the cluster it broke because of different dependency version.

[tonygardella]

@dlebauer @amfox37 This is the script we used to get a reference run working for FATES at BU and the one Shawn uses at BNL: https://github.com/PecanProject/pecan/blob/develop/models/fates/inst/create_CLM5-FATES_1x1br_refrun.sh

[tonygardella]

On CLM 4.5 We ran into a perl issue. Luckily others in the Community were able to work through it here: https://bb.cgd.ucar.edu/cesm-122-porting-error

Essentially there was change in perl where you needed to add parenthesis around the entire function call. foreach my $model qw(cpl atm lnd ice ocn glc rof wav) should be foreach my $model (qw(cpl atm lnd ice ocn glc rof wav))

I doubt that it's an issue now though.

[tonygardella]

@dlebauer This was the machine settings in config_machines.xml that worked on the VM:

<machine MACH="ubuntu">
<DESC>ORNL institutional cluster, phase 2</DESC>
         <OS>LINUX</OS>
         <COMPILERS>gnu</COMPILERS>
         <MPILIBS>mpich,mpi-serial,openmpi</MPILIBS>
         <RUNDIR>/home/carya/clm4_5_1_r085/run/$CASE/run</RUNDIR>
         <EXEROOT>/home/carya/clm4_5_1_r085/run/$CASE/bld</EXEROOT>
         <DIN_LOC_ROOT>/home/carya/ccsm_inputdata</DIN_LOC_ROOT>
         <DIN_LOC_ROOT_CLMFORC>/home/carya/ccsm_inputdata/atm/datm7</DIN_LOC_ROOT_CLMFORC>
         <DOUT_S_ROOT>/home/carya/clm4_5_1_r085/run/archive/$CASE</DOUT_S_ROOT>
         <DOUT_L_MSROOT>UNSET</DOUT_L_MSROOT>
         <BATCHQUERY>qstat</BATCHQUERY>
         <BATCHSUBMIT>qsub</BATCHSUBMIT>
         <MAX_TASKS_PER_NODE>4</MAX_TASKS_PER_NODE>
</machine>

[dlebauer]

Following discussions today, it seems that we are going to focus development on a new package, PEcAn.CTSM that will support CLM5 (CTSM) and also provide helper functions for related models (PEcAn.CTSM_FATES, PEcAn.ELM, PEcAn.ELM_FATES). Can this issue be closed?

[github-actions[bot]]

This issue is stale because it has been open 365 days with no activity.