more aerosol metrics in e3sm_diags

[mahf708]

Description

Add preliminary code for aerosol-related diagnostics from the v3 atmosphere subgroup "Aerosol Working Group" led by Susannah Burrows. This work is partially from a collaborative effort by Naser Mahfouz, Johannes Mülmenstädt, and Susannah Burrows in 2023, and we picked a subset of diagnostics we think will be helpful to a wider audience when evaluating aerosols. This PR adds aerosol masses at different levels, aerosol integrated burdens, aerosol--cloud interaction metrics (e.g., cdnc, lwp), as well as aerosol effective radiative forcing. The PR also exposes all 2D "AeroCom" variables to the derived parameter list (thus enabling easier independent usage by users and/or extending e3sm_diags in future releases).

Caveats:

• The specs in cfg files are not final and will be edited based on feedback from users in upcoming releases.
• Note that the code added may result in non-fatal (harmless) runtime errors (which will appear in e3sm_diags logs) due to potential unavailability of all specified parameters in the h0 tape.

Closes #760

Checklist

• [x] My code follows the style guidelines of this project
• [x] I have performed a self-review of my own code
• [x] My changes generate no new warnings (⚠️ the added code will generate non-fatal runtime errors due to potentially unavailability of some diagnostics in the h0 tapes ⚠️)
• [x] Any dependent changes have been merged and published in downstream modules

If applicable:

• [x] New and existing unit tests pass with my changes (locally and CI/CD build)
• [ ] I have added tests that prove my fix is effective or that my feature works
• [x] I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I have noted that this is a breaking change for a major release (fix or feature that would cause existing functionality to not work as expected)

[mahf708]

Hi @chengzhuzhang, I added basic support for more aerosol variables through the derived variables in acme.py. I will try to refine some aspects of the code (e.g., the cdnc/lwp functions could be improved). Please have a look and let me know if you'd like any additions or fixes. I largely collected this code from previous efforts that occurred over the summer, so I may have done messed something up. The tests pass and figures produced can be found here: https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.ngmahfouz/E3SMv3_dev/test-aero-metrics-in-sets/

Some notes:

1. Not everything added is used immediately; I don't have much time to test figures for all variables now, but hopefully this preliminary support is helpful; I am hoping users will give us feedback about figures and we can fix them as we move forward
2. In the future, if we have more funding for this type of effort, we can bring to life a new set (call it awg metrics or something); for now, we are just plugging things into the easiest and most straightforward interface
3. See the (mostly Susannah's) solution for correcting mass/burden units (without the unit-change hack, numbers won't even show up on these figures); hope it is acceptable, but in the future, we likely can do better than that

For reviews, I cannot request review from anyone, so please invite Johannes (@jmuelmen) and Susannah (@susburrows) to review this PR. They contributed to this work throughout the year (and I added them as co-authors in two important commits above where their contributions were especially important).

[jmuelmen]

These diagnostics will be extremely useful for ACI analysis. Great to have them in e3sm_diags!

[susburrows]

Thanks, @mahf708, for implementing the changes in this PR and following through on the steps required to submit them to the e3sm_diags repository. These diagnostics are extremely valuable for understanding and tracking how the aerosol forcing and understanding the 3D distribution of aerosols change across model versions, while keeping the I/O burden manageable (e.g., by using 2D fields such as cross-sections of mass concentrations at several pressure levels).

Thanks for providing the opportunity for these contributions from the AWG to be included into e3sm_diags, @chengzhuzhang .

I think this looks good and am approving, based on examination of the test output. If further changes are required in the future, I trust that those can be addressed in separate PRs.

[mahf708]

On a side-note, I started refactoring acme.py in another branch. I split it into derivations.py, formulas.py, and utils.py. These modules operate on Xarray objects instead of cdms2 objects. Also the derived_variables dictionary will eventually be cleaned up because it is massive and hard to follow.

This is great! I intentionally put my additions at the end because I knew this was going to be refactored. I also think we may end up revisiting my haphazard implementation soon. Once, you're ready with your branch, I can push my additions to it, so that you don't have to bother rewriting this code ;)

Also, great to have this done via xarray (I know xarray, but I know nothing about cmds2). I actually was wondering how I could do a better cutoff for these quantities divided by lcc/icc (obviously these can be zero, so we need a cutoff; turns out it worked well without a cutoff for cdnc, but not so well for lwp). Instead of dwelling on it, I thought I would revisit it later :)

[tomvothecoder]

This is great! I intentionally put my additions at the end because I knew this was going to be refactored. I also think we may end up revisiting my haphazard implementation soon. Once, you're ready with your branch, I can push my additions to it, so that you don't have to bother rewriting this code ;)

Also, great to have this done via xarray (I know xarray, but I know nothing about cmds2).

Awesome! Whenever you're ready, you can open a PR that includes the Xarray version of the code. You just need to cdat-migration-fy24 as the base branch.

This PR looks good to me.

[mahf708]

Okay, @tomvothecoder feel free to merge and initiate a release candidate for @xylar to start testing the emergency e3sm-unified release. I just rebased this PR.