E3SM-Project / e3sm_to_cmip

Tools to CMORize E3SM output
https://e3sm-to-cmip.readthedocs.io/en/latest/
MIT License
7 stars 7 forks source link

[Feature]: Consolidate CMOR setup #273

Closed tomvothecoder closed 2 months ago

tomvothecoder commented 3 months ago

Is your feature request related to a problem?

CMOR SETUP CONSOLIDATION:

The motivation for cmor.setup reconfigurations is that this cmor function was called from 7 separate locations:

Aside from unnecessary replication of codes, each call to cmor.setup invokes a cmor-internal "logger" setup, whose only external accessibility is the name of the logfile to supply. There is no ability to alter its propagate(True/False), date-format or other attributes. (I believe it is responsible for the "duplicated log messages" problem, as only those messages with the CMOR date-format appear to be duplicated.)

Describe the solution you'd like

Now, there are only TWO functions that invoke cmor.setup: util.setup_cmor(), and handler._setup_cmor_module(). The latter is applied to every handler managed through the "handlers.yaml" construct. The former handles all other variables. The function in mpas.py has been eliminated.

Describe alternatives you've considered

No response

Additional context

No response

tomvothecoder commented 3 months ago

Pulled this from #261

TonyB9000 commented 3 months ago

@tomvothecoder Thank you, Tom, for setting this up. I tend to do all of my editing (and git-actions) from the console command-line, and find it difficult to coordinate a mix of console and web-based operations. I will attempt to forge branches off of the former "Consolidate cmor setups and logging" branch, but reverting selected changes (commits? pushes?) is something I will need to research. Something about "revert " for selected hashes. Must review these.

tomvothecoder commented 3 months ago

@TonyB9000 An alternative and possibly easier/hackier way is to look at the Files Diff for your PR #261, group the related changes by module, then remove the unrelated changes manually (e.g,. remove logger changes from the cmor setup refactor branch).

You can create sub-branches off of the branch in #261.

TonyB9000 commented 2 months ago

This issue has been addressed by the merge on issue #273 #275. The only remaining variable is that in the call to "cmor.setup" one supplies a logfile (independent of the e2c logger setup), and that logfile is comprised of a directory (default: PWD) and a filename derived from the variable being cmorized. I believe this cmor.setup call (internally) changes the global log-date format, and I have found no way to avoid this.

I hereby close this issue.