Add ability of mksurfdata_esmf to create datasets that can be used for Paleo work...

ekluzek commented 5 years ago

We want to have some customized files in place, in order to allow mksurfdata_esmf to create surface datasets that can be used for Paleo work for CLM6.0. The Paleo working group will support these specific datasets. A specific development project for this is the Last Glacial Maximum (LGM). There was work on this in CLM4.0 for CESM1.2.2, but we need to extend this to the latest ctsm and for CLM6.0. So there are some modern raw datasets that were added in that would need to changed to correspond to paleo conditions.

This won't be an out of the box feature. So there won't be a simple option to do this. But, the user can modify the namelist that gen_mksurfdata_namelist creates and put in the specific Paleo datasets that are supported by the Paleo group, and then create a surface dataset that way, customizing a batch script to submit it on Derecho.

ekluzek commented 5 years ago

A CLM4_5 dataset that Bob Tomas created is here:

/glade/p/cesm/palwg/lgm_cesm2.0/surfdat/surfdata_0.9x1.25_16pfts_simyr21ka_fixedlakes_c161219.nc

Note it used these special files:

    :Input_grid_dataset = "map_0.5x0.5_LGM_to_0.9x1.25_nomask_aave_da_c160411.nc" ;
    :Inland_wetland_raw_data_file_name = "mksrf_lanwat_lgm21ka.nn.101129.nc" ;
    :Glacier_raw_data_file_name = "mksrf_glacier_lgm21ka.nn.160503.nc" ;
    :Urban_Topography_raw_data_file_name = "mksrf_topo.10min.lgm21k.101129.nc" ;
    :map_pft_file_name = "map_0.5x0.5_LGM_to_0.9x1.25_nomask_aave_da_c160411.nc" ;
    :map_wetlnd_file = "map_0.5x0.5_LGM_to_0.9x1.25_nomask_aave_da_c160411.nc" ;
    :map_glacier_file = "map_0.5x0.5_LGM_to_0.9x1.25_nomask_aave_da_c160411.nc" ;
    :map_soil_color_file = "map_0.5x0.5_LGM_to_0.9x1.25_nomask_aave_da_c160411.nc" ;
    :map_soil_organic_file = "map_0.5x0.5_LGM_to_0.9x1.25_nomask_aave_da_c160411.nc" ;
    :map_urban_file = "map_0.5x0.5_LGM_to_0.9x1.25_nomask_aave_da_c160411.nc" ;
    :map_fmax_file = "map_0.5x0.5_LGM_to_0.9x1.25_nomask_aave_da_c160411.nc" ;
    :Soil_color_raw_data_file_name = "mksrf_soilcol_lgm21ka.nn.160506.nc" ;
    :Fmax_raw_data_file_name = "mksrf_fmax_lgm21ka.nn.101129.nc" ;
    :Organic_matter_raw_data_file_name = "mksrf_organic.10level.0.5deg_lgm21ka.nn-avg.110107.nc" ;
    :Vegetation_type_raw_data_filename = "mksrf_pft_lgm21ka.harvest.nn.101129.nc" ;

Now it also overwrote the following variables from the standard setup: TOPO_GLC_MEC, PCT_GLC_ICESHEET, PCT_GLC_GIC, PCT_GLC_MEC_GIC, PCT_GLC_MEC, GLC_MEC and then zero's lakes where glacier's were with:

/glade/p/cesm/palwg/lgm_cesm2.0/surfdat/rm_lakes_where_icesheet.ncl

ekluzek commented 3 years ago

Adding @negin513 and @slevisconsulting to this issue as it's related to some of the work they are doing on getting this process working for WRF. Some of the solutions they are working on there will also help out here with Paleo raw datasets.

slevis-lmwg commented 3 years ago

Thanks @ekluzek In the next few weeks and months we will post updates to the process of generating CTSM surface datasets here: https://github.com/ESCOMP/ctsm/issues/644

ekluzek commented 2 years ago

We should evaluate this for the new mksurfdata_esmf and modify fsurdat tools .

slevis-lmwg commented 1 year ago

We should evaluate this for the new mksurfdata_esmf and modify fsurdat tools .

A few thoughts that I wanted to put in writing here: Paleo applications tend to vary in their needs and, therefore, in their use of the tools. The Last Glacial Maximum is an example where many things change (PFTs, continental outlines, lakes, ...). The new mksurfdata_esmf presumably works fine with the LGM raw datasets listed above. The fsurdat_modifier tool may prove helpful in making simple idealized changes to land areas, including where continental outlines have changed.

It's important to remember though that changes to the continental outlines do not take effect unless the mask_mesh/ocn_mesh file gets updated (in env_run.xml). For F and I cases one can accomplish this with the mesh_mask_modifier. For B cases Alper is putting together a tool for the CSSI project that should address that need.

slevis-lmwg commented 1 year ago

@ekluzek and @slevis-lmwg going over ctsm5.2 project board: This issue for Paleo (#578) seems more of an application of the mksurfdata_esmf tool that we can explain in the documentation (#1718).

ekluzek commented 4 months ago

This will be a post ctsm5.2.0 activity and we should get with the Paleo group to make sure mksurfdata_esmf works for them.

wwieder commented 4 months ago

From the CESM project meeting, it looks like @sophmaca will be our paleoclimate contact when we circle back to this. I also seems like @jiang-zhu may like this to be something we need to produce before the "June" code freeze.

wwieder commented 4 months ago

For now I'll assign this to @slevis-lmwg, as he's our surface data expert with CTSM5.2.

slevis-lmwg commented 4 months ago

Adding notes from the 2024/3/12 CESM Proj. Meeting (and side conversations):

The paleo group will need to reuse (likely with modifications) or regenerate raw datasets as inputs to the mksurfdata_esmf tool to generate an fsurdat file for their LGM simulations.
Isla pointed out that the fsurdat_modifier and/or mask_modifier tools may prove helpful in this work. The former allows relatively simple modifications to fsurdat files. The latter may help with the change of continental outlines, though it sounds as though changing the continental outlines may raise difficulties with other models (MOM?).

ekluzek commented 4 months ago

I updated the introduction to move from what you do in mksurfdata_map to what you do in mksurfdata_esmf. I think those instructions might be sufficient for what's needed here. I'm thinking just having the instructions on what to do might be sufficient?

The three things beyond the instructions would be:

Trying the instructions out for LGM to make sure it will work
To make it easier to do with some type of usernl* mechanism or something
To support LGM datasets and an easy way to create namelists for it with mksurfdata_esmf

We should at least do 1, but I'm not sure if we can (or should) do 2 and 3 now. But, it would be good to have that support in the long run.

wwieder commented 4 months ago

Thanks @ekluzek I think it may be a good idea to create an LGM compset at paleo-friendly resolution, as it seems like @dlawrenncar wants to be able to test ECS with this configuration before the CESM3 release.

After we do this, hopefully is clear enough what raw datasets paleo users would need to change to create their own surface datasets for other periods, but I agree, the more complicated configurations are something we don't need to support out of the box.

sophmaca commented 4 months ago

I updated the introduction to move from what you do in mksurfdata_map to what you do in mksurfdata_esmf. I think those instructions might be sufficient for what's needed here. I'm thinking just having the instructions on what to do might be sufficient?

The three things beyond the instructions would be:

Trying the instructions out for LGM to make sure it will work

To make it easier to do with some type of usernl* mechanism or something

To support LGM datasets and an easy way to create namelists for it with mksurfdata_esmf

We should at least do 1, but I'm not sure if we can (or should) do 2 and 3 now. But, it would be good to have that support in the long run.

@ekluzek I'm happy to carry out/test 1. with some preliminary instructions

slevis-lmwg commented 4 months ago

@sophmaca here are prelim. instructions:

[x] In https://github.com/ESCOMP/CTSM/discussions/1919, complete step "E) git checkout ctsm5.2.mksurfdata".
[x] In step F) execute cd tools/mksurfdata_esmf and follow the instructions in the README there, rather than the ones in #1919. Select the --nocrop --potveg_flag when running the ./gen_mksurfdata_namelist script. By the end of this README you should have a "no anthro" fsurdat file.
[ ] Once you complete (1) and (2) (reach out to me and/or @ekluzek if you get stuck), we will discuss how to introduce the LGM data. E.g. we may repeat (2) with a modified namelist, where you point to existing (likely modified) raw datasets and/or you may generate new raw datasets...

slevis-lmwg commented 4 months ago

Linking new issue #2420 uncovered by @sophmaca: mksurfdata_esmf puts urban in an fsurdat intended as PtVeg.

slevis-lmwg commented 4 months ago

@sophmaca for when you try this again (issue #2420 should be fixed by the end of the week): I updated the "preliminary instructions" above to include another flag along with potential veg. While not required, this will make the fsurdat file smaller and the model may run more efficiently.

sophmaca commented 4 months ago

@slevis-lmwg I repeated the steps with additional flag and the "no anthro" fsurdat looks good to me!

wwieder commented 4 months ago

Thanks for testing this out, Sophia & thanks for helping @slevis-lmwg.

@ekluzek what's the definition of done for this issue? Does it come down to documentation at this point?

slevis-lmwg commented 4 months ago

@wwieder I will change the numbers in my post above to checkboxes to indicate TODOs for this issue.

slevis-lmwg commented 4 months ago

So @sophmaca the next step is to manually change the namelist to use LGM raw datasets. You may be able to use or start from existing LGM raw datasets, or you may generate new ones from scratch. If you need help with this step, I think that we should meet to discuss.

ekluzek commented 4 months ago

@wwieder @slevis-lmwg is correct the immediate thing is to finish his checkboxes with @sophmaca. So she will add in LGM data and see what happens when she does. I expect there to be issues to work out.

After that is done, we need to make sure the documentation is handled. And then I think that's good for now. There are two things that we could do above

https://github.com/ESCOMP/CTSM/issues/578#issuecomment-1992156788

that we probably won't do now until we have more time. So we should close this when we've worked out the process and have documentation on it. And then will open new issues as appropriate when we decide if (and how) we want to make the process easier to do.

sophmaca commented 4 months ago

@wwieder @ekluzek @slevis-lmwg I will discuss the LGM surface characteristics we want to include and existing LGM raw datasets with the paleo group and get back to you all

sophmaca commented 4 months ago

@wwieder @slevis-lmwg is correct the immediate thing is to finish his checkboxes with @sophmaca. So she will add in LGM data and see what happens when she does. I expect there to be issues to work out.

After that is done, we need to make sure the documentation is handled. And then I think that's good for now. There are two things that we could do above

#578 (comment)

that we probably won't do now until we have more time. So we should close this when we've worked out the process and have documentation on it. And then will open new issues as appropriate when we decide if (and how) we want to make the process easier to do.

@ekluzek @slevis-lmwg Question regarding producing LGM raw datasets: do we still need to regrid everything to 0.5x0.5 with the new mksurfdata_esmf tool?

slevis-lmwg commented 4 months ago

That's not a requirement.

ekluzek commented 4 months ago

@sophmaca the LGM data can come in on any resolution. The only requirement is that we have a mesh file that goes along with the same grid. 0.5x0.5 was the grid that Paleo used in the past, but it could be any other grid resolution. And if you have a file with data we have various ways to create a mesh file from it.

ESCOMP / CTSM

Add ability of mksurfdata_esmf to create datasets that can be used for Paleo work... #578