Implementing Marine Organic Aerosols to CESM2 #531

[XiZhao1013]

We expand the Modal Aerosol Module (MAM) in CESM2-CAM6 to consider the emission of marine organic aerosol (MOA) as well as its impacts by acting as cloud condensation nuclei (CCN) and ice nucleation particles (INPs).

MOA is an important source for CCN and INPs over the Southern Ocean and has shown important climate implications over the remote oceans, which are detailed in this recent paper (Zhao and Liu et al, ACP, 2021).

There is a general interest in MOA in the community. We would like to share our MOA code in CAM6 with the community, which can be found here.

Summary of our code developments:

• added emission of MOA
• added CCN and INPs effects of MOA
• Code development is currently based on the cesm2.0.1-rc.01

Thank you in advance to @lkemmons for helping with reviewing our changes.

[nusbaume]

I was told by @cmcclus that she will be opening the PR for this, so am assigning this issue to her.

[adamrher]

Is MOA going to be a brand new tracer in the model? Or is it just a biogenic property added to an existing aerosol tracer?

[XiZhao1013]

Is MOA going to be a brand new tracer in the model? Or is it just a biogenic property added to an existing aerosol tracer?

MOA is going to be a new tracer. Since it has different aerosol properties(density, hygroscopicity, nucleation ability) compared with existing aerosol species.

[cmcclus]

will be two new aerosol tracers - Aitken and Accumulation modes

[XiZhao1013]

will be two new aerosol tracers - Aitken and Accumulation modes

Thanks, Christina. One comment here, MOA is emitted to Aitken and Accumulation modes with the assumption of internally mixed with sea salt (more information cloud be found in Zhao et al., ACP, 2021; Burrows et al., 2018).

[adamrher]

Still a little confused. Mind you I am just starting to get my head around all the tracers in the model. There are currently three sea-salt tracers: ncl_a1, ncl_a2, ncl_a3. Are we adding more? Or just enhancing these three existing ones?

[XiZhao1013]

Still a little confused. Mind you I am just starting to get my head around all the tracers in the model. There are currently three sea-salt tracers: ncl_a1, ncl_a2, ncl_a3. Are we adding more? Or just enhancing these three existing ones?

We are adding more, we will add mom_a1, mom_a2, and mom_a4.

[XiZhao1013]

I have implemented the MOA into the development branch. But each time when I output MOA (mom_a1, TMmom_a1...), the writing process takes more than 1 hour (the MOA can be output, but very slow). I will be very appreciative if anyone could give any suggestion on this. @cmcclus @nusbaume

[gold2718]

@XiZhao1013, can you include a CAM fork + branch here? I might be able to spot something that is increasing the history output time. Alternatively (or in addition), please post a case directory on an accessible machine (e.g., Cheyenne).

[XiZhao1013]

Thanks for your suggestion, it is very instructive for a new user of GitHub @gold2718.

the branch name: MAM4_MOA CAM fork: XiZhao1013 / CAM (not sure if I provide the right information)

Case directory on Cheyenne: /gpfs/fs1/work/xizhao/cesm2_develop_branch_new/cases/cesm2_mom_new_4 Model output: /gpfs/fs1/scratch/xizhao/test_case/cesm2_mom_new_4/run

[gold2718]

You are writing to /gpfs, can you try writing to /glade/scratch instead? That is much faster in most cases (especially from the compute nodes). Also, you are using a very old format for user_nl_cam, is there a need for that?

[XiZhao1013]

@gold2718 Thanks. I will try to write to /glade/scratch. There is no need to use the old format for user_nl_cam, I just copy my old script. Could you point me to a new format for user_nl_cam?

[XiZhao1013]

I have added MOA to the CESM development branch successfully, and ran a two-month test, the result looks good to me.  However, I noticed an output issue, the model needs 2 hours to write a monthly mean output after adding MOA code. (the model can output, but needs a lot of time) This issue is more related to the code structure and is beyond my knowledge of MOA. Any help and suggestion will be appreciated.

additional information of the code: mode code: /gpfs/fs1/work/xizhao/mom_develop/CAM_2 case dir: /gpfs/fs1/work/xizhao/mom_develop/cases/cesm2_mom_dep_CAM2 output dir: /gpfs/fs1/scratch/xizhao/test_case/archive/cesm2_mom_dep_CAM2

code on github: https://github.com/XiZhao1013/CAM/tree/moa

[gold2718]

What machine are these directories on?

[XiZhao1013]

on Cheyenne @gold2718

[gold2718]

I don't see the model output taking an unusual amount of time. How much time does the same setup without MOA take? Normally for a 1-degree FV run, we use 360 PEs on Cheyenne, not 36.

[XiZhao1013]

@gold2718 Appreciate your help! I will try 360 PEs setting later.

I attached a plot, after the model finished calculation on 13 Jan, it took 2 hours to write the monthly mean file for Jan. Please find the code diff at: https://github.com/ESCOMP/CAM/compare/cam_development...XiZhao1013:CAM:moa

One potential cause is in: /gpfs/fs1/work/xizhao/mom_develop/CAM_2/src/chemistry/utils/tracer_data.F90 I added "subroutine put_fld_data" (needed by src/chemistry/modal_aero/seasalt_model.F90), this subroutine is removed in the new version of CESM, but is exist in the previous version. may this be the reason?

[gold2718]

I am not familiar with put_fld_data, however, I can't even figure out what it is doing (looking at your branch). I think the difference in the timestamps is due to the run time, not in model output. That should improve performance. Putting some timer calls around your new code would help understand the impact of your modifications (we could look in the timing files).

[XiZhao1013]

@gold2718 I set up a new simulation with 360 PEs as you suggested.

half an hour ago, the model begin to output the first file, and is still outputting now (see the attached plot). is it normal to take more than half an hour to write a file with 5 variables?

[gold2718]

File access times are an unreliable method for analyzing model performance for several reasons. For instance, log output is buffered and write times do not closely follow model command execution. Also, CAM history will sometimes access history files at times other than when doing the actual write (can be on a later timestep). I think the best approach at this stage is looking at model timing files with and without your modifications running. Also, comparing a "without" run between your branch and a clean checkout of cam6_3_078 (the tag your branch is based on) would be very helpful.