Need ability to link to MICM library at build time

[nusbaume]

What is the feature/what would you like to discuss?

CAMDEN needs the ability to link to the Model Independent Chemical Mechanisms (MICM) chemistry package.

For the MICM-equivalent issue see: https://github.com/NCAR/musica/issues/21

Is there anyone in particular you want to be part of this conversation?

@mattldawson

Will this change (regression test) answers?

No

Will you be implementing this enhancement yourself?

Any Software Engineer can do this

[nusbaume]

Also pinging @K20shores

[cacraigucar]

However this is implemented, we need to be able to "freeze" MICM versions and only update them when we are ready. This would be similar to what we do with the Externals.cfg file via tags.

[K20shores]

The intent is to create an environment module that can be loaded prior to building camden. This will allow multiple versions of micm to be present and would not force camden to use any one particular version.

[nusbaume]

I imagine too we'll need to provide documentation or techniques to help a user install MICM on their own machine so that it can be used with a ported version of CAMDEN/SIMA.

One solution could be to add a special flag to the CAMDEN build options that will build MICM from source using the same compiler/libraries that the rest of SIMA is built with if the user wants. Then after that they can just link to that already built version of MICM for all future SIMA simulations.

Of course, I think the first goal should be to just get this all working on Cheyenne. We can worry about all of the porting issues later.

[K20shores]

MICM already supports being built as a library and can be installed on a personal machine. There is not really a reason to build MICM from source each time CAMDEN is built, but it is certainly entirely possible.

[sjsprecious]

One related question: if the MICM library is built with different compilers & flags that are used to build CAMDEN, will it just work fine or it will crash somewhere? If it is the latter case, then it seems necessary to provide the option of building MICM from source code. Otherwise, we may have to force the users to use the same combination of compiler and flags for MICM & CAMDEN.

[nusbaume]

@K20shores Agreed building it wouldn't be the ideal solution, I was just trying to think of how to make it easiest on a user who has limited computing experience who needs to bring SIMA over to a machine that they do not have elevated privileges on.

Then again we expect users to have already built things like NetCDF ahead of time so maybe MICM should be no different.

Anyways, I definitely don't have all the answers now, I just wanted to write that thought/concern down here first before I completely forgot about it.

[K20shores]

MICM can always be built from source. That will never change.

It is a good question about the compiler flags and what is different. As Jesse said, netCDF is a required pre-installation, so I imagine the behavior would be the same there. It's likely only difference in debug information that matter, so long as the ABI is the same. The user would be responsible for selecting the correct compiler for prerequisite packages and the build of CAMDEN.

To me, the most user friendly thing would be module use-micm and CAMDEN would attempt to include MICM in the build and fail if libraries and header files aren't found. Perhaps a solution is to create environment modules for each supported compiler on our HPC machines for each version of MICM we release? This could be the fastest option for most users, but for anyone who needs more control they can always go back to building from source.

[sjsprecious]

Thanks @K20shores. The method you proposed sounds good to me and I think it is doable at other HPC centers, too. We may just need to document the details somewhere so that a user can refer to them later.

[K20shores]

Another option is our MUSICA library. It's setup to build every part of MUSICA (right now, that's TUV-x and MICM). So, instead of building MICM, we could build MUSICA with the flag that includes the build of MICM. In this way, any new components to MUSICA will be buildable in CAMDEN automatically, but not useable until someone writes the CCPP wrapper codes.

[K20shores]

To make it clearer, Matt made this diagram. Blue stuff will be an environment module and prebuilt. Red stuff will be build whenever CAMDEN is built.

[gold2718]

Then again we expect users to have already built things like NetCDF ahead of time so maybe MICM should be no different.

And ESMF!

Two quick comments (which may be obvious to everyone already) and a question: 1) For supported macines, the modules or paths for the various MUSICA/MICM various libraries (compiler, debug level etc.) can just go into the config_machines.xml file which should ease use. 2) For unsupported machines, the library options can be added to the 'container' machine type (just need to get new images built).

For the users that have to build the libraries, Spack seems to be gaining quite a bit of popularity. Is MUSICA/MICM going to support building with Spack?

[K20shores]

@gold2718 we don't have plans to support Spack but that's because I think we only became aware of it with your comment. It IS our goal to make our software available through different package managers (homebrew, and various Linux package management distributions). If Spack fills the gap for HPC machines then we will heavily consider this. Thanks for the suggestion!

[K20shores]

Closed by #225 225