Create field_table on the fly

climbfuji commented 3 years ago

Description

The field_table file is required in the run directory and used early on in the model run to set up the tracers for the run. This file contains information that is 100% determined by the choice of microphysics and microphysics-dependent options, if any. Inconsistencies often result in difficult to track down segmentation faults.

We should therefore strive for a way to generate the field_table file on the fly instead of requiring it in the run directory and be consistent with the microphysics settings.

Solution

Generate field_table depending on the choice of microphysics and microphysics-dependent options, if any, before calling the FV3 dycore atmosphere_init routine. At this point in atmos_model.F90, the contents of input.nml is already known (when using INTERNAL_FILE_NML at least, I don't think the other option is still supported and/or tested). The challenge is that the GFS_control DDT routine that reads the internal namelist cannot be called yet, because it depends on information from the dycore init - a classical chicken and egg problem.

I see several options to solve the problem:

Before starting the model, run a small (Python) script that checks for the appropriate microphysics options in the suite definition file and input.nml, and generates field_table
- advantage: easy to code up because independent and because Python is perfectly suited for working with text files
- disadvantage: yet another thing that needs to be remembered and called as part of the workflow / rt.sh
In the model, before calling atmosphere_init, use a Fortran subroutine that parses the SDF / namelist and generates field_table
- advantage: all Fortran, and all in one code
- disadvantage: Fortran is as well suited for parsing text files as an F350 for shopping in downtown Denver (and I owned one of these, so I know what I am talking about)
In the model, before calling atmosphere_init, call a Python program from Fortran that parses the SDF / namelist and generates field_table
- advantage: all in one code, and use the best tool for the given task
- disadvantage: calling Python from Fortran isn't straightforward and introduces additional dependencies (e.g. CFFI)
Modify FMS such that it parses input.nml instead of field_table to set up the tracer manager
- advantage: none of the sub-ideal solutions above needed
- disadvantage: need to add code to FMS that runs counter the design of FMS
Augment the CCPP framework to do the parsing/information collettion as part of the CCPP init step (not the CCPP physics init step). Move the CCPP init step (which currently happens just before CCPP physics init) up so that it runs before atmosphere_init. Provide a query function for the microphysics settings and generated field_table based on this information
- advantage: CCPP knows at build time which microphysics are used in which suite definition file - the microphysics option query function can thus be auto-generated at build time, a simple Fortran call suffices to obtain the information in atmos_model.F90 before calling atmosphere_init
- disadvantage: CCPP doesn't know at build time about other (currently runtime) options such as whether the aerosol-aware or non-aerosol-aware version of Thompson is used, or whether MG3 runs with or without graupel. An input.nml query would still be required unless we come up with a design that provides all this information at build time (e.g. encoding such important namelist switches in the suite definition file instead of input.nml)
  - advantage: a clear separation of who provides information about what physics are run (CCPP) and how to translate this into model-specific data (fv3atm -> field_table)

All in all, the CCPP-based solution seems the cleanest and easiest to me, even though it means that one would lose the option to switch between different flavors of the same microphysis at run time. (But how often was that needed in the past?)

Alternatives

I am looking forward to learn about alternative solutions from all the smart developers out there.

yangfanglin commented 3 years ago

We probably should talk to the NCAR FV3 team (Steve Goldhaber, Peter Lauritzen etc) to learn how they are generating the field_table "on the fly". They worked on reengineering the field_table a couple of years ago. I'd also suggest including GFDL's Rusty Benson in the discussion. This is not a purely physics issue. I believe tracer transport uses this table as well.

SMoorthi-emc commented 3 years ago

After reading all these options, it appears that the current option is not so bad at all. Currently, the user knows what field table to use and if the user/developer adds a new scheme, knows how to update. When it is magically created within the code, the developer of a new scheme will have to dive into the code. I hope the option to provide an external field_table will be maintained (it is easier to switch field tables in the run script depending on the scheme) even if it is generated optionally within the code.

On Mon, Jun 28, 2021 at 11:33 PM Fanglin Yang @.***> wrote:

We probably should talk to the NCAR FV3 team (Steve Goldhaber, Peter Lauritzen etc) to learn how they are generating the field_table "on the fly". They worked on reengineering the field_table a couple of years ago. I'd also suggest including GFDL's Rusty Benson in the discussion. This is not a purely physics issue. I believe tracer transport uses this table as well.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-weather-model/issues/673#issuecomment-870203578, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALLVRYXSOMXL2CCN7LDVCUDTVE5G7ANCNFSM47PGBS7Q .

-- Dr. Shrinivas Moorthi Research Meteorologist Modeling and Data Assimilation Branch Environmental Modeling Center / National Centers for Environmental Prediction 5830 University Research Court - (W/NP23), College Park MD 20740 USA Tel: (301)683-3718

e-mail: @.*** Phone: (301) 683-3718 Fax: (301) 683-3718

climbfuji commented 3 years ago

I had briefly talked with Rusty about this, but even taking it one step further (use an INTERNAL_FIELD_TABLE logic, similar to INTERNAL_FILE_NML). This however involves quite some changes in FMS, which I would like to avoid. And it is an independent step that can be taken after we implemented the logic. And yes, I also talked with Steve Goldhaber about this in our CCPP framework developer meeting. But I'll check again with him about the approach the use, good point.

On Jun 28, 2021, at 9:33 PM, Fanglin Yang @.***> wrote:

We probably should talk to the NCAR FV3 team (Steve Goldhaber, Peter Lauritzen etc) to learn how they are generating the field_table "on the fly". They worked on reengineering the field_table a couple of years ago. I'd also suggest including GFDL's Rusty Benson in the discussion. This is not a purely physics issue. I believe tracer transport uses this table as well.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-weather-model/issues/673#issuecomment-870203578, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB5C2RMBPKCPKQKICARHWTLTVE5G5ANCNFSM47PGBS7Q.

ufs-community / ufs-weather-model