need better way to find all possible history fields

[billsacks]

Bill Sacks < sacks@ucar.edu > - 2015-06-26 11:29:48 -0600 Bugzilla Id: 2187 Bugzilla CC: andre@ucar.edu, dlawren@ucar.edu, mvertens@ucar.edu, rfisher@ucar.edu,

Danica raised the point that it is hard to find all possible history fields with the new CLM code organization. I know we have findHistFields.pl, but it doesn't seem to work right from the latest code base: it finds a lot of BGC-related fields, but didn't find the fields we were looking for (RH and QBOT in atm2lndType.F90).

So maybe this tool just needs to be fixed? Or maybe I was using it wrong.

We also felt that this should either (a) print more information about the fields (e.g. whether it's active or inactive by default) or (b) print the file in which a given field is defined, so that you can go there to find more information about the field.

In looking through this code, I also noticed that the list of files to search is hard-coded. It seems like this list will quickly get out-of-date, if it isn't already. Can it simply search all F90 files below src?

Alternatively, if this tool is too hard to maintain, perhaps another option would be to give a namelist option to CLM to make it write out all history fields to its log file in initialization.

[billsacks]

Bill Sacks < sacks@ucar.edu > - 2016-10-24 14:33:02 -0600

Feeling from today's clm-cmt meeting is that, at least for now, we may just make use of the runtime output for this purpose. We think there's already something like this; it may just be a matter of cleaning up this output to be easily greppable (e.g., one line per variable).

But this depends somewhat on what the purpose is: e.g., the runtime output can't tell you what file the field is defined in (need to do a later grep for that), or the association between variable names in the code and history file variable names.

So we should ask Danica what her main use case is.

[billsacks]

@danicalombardozzi I'm closing this old issue. Feel free to comment further and reopen if you'd still like something more here.

I definitely see the value in this, I'm just not sure this is high enough priority to get done in the foreseeable future... but I'm happy to hear any further thoughts you may have.

[billsacks]

It seems like this is still wanted, e.g., for #941 . I still feel that the best way to accomplish this is to run a short case in the desired configuration and then either parse the log file output (which may require tweaking that output to make it greppable and possibly adding some additional information in the output) or possibly pull it off of the header information from a history file (probably in conjunction with #29 ). The old findHistFields.pl script (which can give you this information without running a case) seems like a maintenance issue, for example requiring code to handle each set of dynamically-generated history field names, of which we have more and more – for CN Product pools, water isotopes, etc – so I suggest ditching that and coming up with something new.

We could potentially (ab)use the system test infrastructure to write a python-based system test that runs a short case with any appropriate namelist flags set then parses the output into the correct format. This wouldn't need to be part of any test suite, but would allow us to generate the desired history file list with a single create_test command.

Blocks #941 .

[billsacks]

@slevisconsulting and I just talked about this. After talking through how we'd do this in python, we realized a much easier approach would be to modify the hist_printflds Fortran subroutine or make a new subroutine like it, so that: If a given namelist flag is set, then it opens a new file for output and outputs the field list in an appropriate format – maybe even directly in the format of a restructuredText table.

[slevis-lmwg]

modify the hist_printflds Fortran subroutine or make a new subroutine like it, so that: If a given namelist flag is set, then it opens a new file for output and outputs the field list in an appropriate format – maybe even directly in the format of a restructuredText table.

I have the code generating the new text file and am evaluating the outcome with this test: ERP_D_Ld5.f10_f10_musgs.IHistClm50BgcCrop.cheyenne_intel.clm-allActive

Before I format as a restructuredText table, I would like to request feedback (@billsacks @wwieder @danicalombardozzi @dlawrenncar @katiedagon ) on the type of information that we should include in the table. Here's a list of options from a sample variable appearing in the file:

VALUE --- VARIABLE NAME AND/OR DESCRIPTION 1234 (history field number; this happens to be the last var in the file generated by this test) LATBASET (name 64 char) latitude vary base temperature for gddplant (long_name 199 char) degree C (units 199 char) pft pft unset (type1d 16 char, type1d_out 16 char, type2d 16 char) 1 1 1234 (numdims, num2d, hpindex) unity unity unity (p2c_scale_type 32 char, c2l_scale_type 32 char, l2g_scale_type 32 char) A A A A A A A A A A (avgflag(:) I haven't looked into why this is an array)

[slevis-lmwg]

The clm4.5 table of history fields includes only name, long_name, and units.

[dlawrenncar]

I'm having a hard time understanding that list fully, but I think the CLM4.5 table is pretty good. The only thing I can imagine adding might be whether it is a pft, column, landunit, or gird variable, though including that might also add confusion since labeling something as a PFT level variable tells user what the underlying representation is, but if it is generally averaged up to grid cell, then I could see a user being confused. So, I guess I might lean towards just sticking to what was listed in CLM4.5, though happy to hear from others.

On Sun, Nov 8, 2020 at 10:43 PM Samuel Levis notifications@github.com wrote:

The clm4.5 table of history fields https://www.cesm.ucar.edu/models/cesm1.2/clm/models/lnd/clm/doc/UsersGuide/history_fields_table_45.xhtml includes only name, long_name, and units.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ESCOMP/CTSM/issues/34#issuecomment-723770765, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFABYVDYPZG47XYMBDUB7NDSO56QNANCNFSM4EIRY2PQ .

[katiedagon]

Something like the CLM4.5 table of history fields for CLM5 / CTSM would be useful to include in the users guide, which was the original motivation behind opening #941

[danicalombardozzi]

I agree that the variable name, long name, and units are the most useful. I see Dave's point about the level of the variable (p, c, g), but I agree that it will likely be confusing since we are writing out grid and PFT-levels more consistently for several variables. Having this basic information gives people enough information about which variable they might want to look for/analyze, and then they can find the appropriate history files and get the remaining information there.

On Mon, Nov 9, 2020 at 10:29 AM Katie Dagon notifications@github.com wrote:

Something like the CLM4.5 table of history fields for CLM5 / CTSM would be useful to include in the users guide, which was the original motivation behind opening #941 https://github.com/ESCOMP/CTSM/issues/941

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ESCOMP/CTSM/issues/34#issuecomment-724159451, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGHW2QMSUCLYVH2MOKFP5JDSPARHVANCNFSM4EIRY2PQ .

-- Dr. Danica Lombardozzi she/her/hers Terrestrial Sciences Section Climate and Global Dynamics National Center for Atmospheric Research Boulder, CO 80305

email: dll@ucar.edu office: (303) 497-1777