Add a tier2 site config for AWS AMIs

[eap]

We would like to have an AWS AMI cable of running multiple compiler toolchains. Ideally we would like to run gnu, clang and intel from a single base image with simple environments scripts and modules to select the proper compiler toolchain.

This will initially be checked in as a tier2 site config but we may upgrade it to a tier1 config with the 1.9 release, especially if this is adopted as JCSDA's preferred internal development environment (an option we are considering).

[climbfuji]

have a look at the ci instance - it runs gnu, intel classic and intel oneapi already, and it should take very little effort to turn this into a ready-to-use ami (and add clang)

[eap]

The CI instance has quite a bit of customization on it already related to the CI setup making it a poor candidate for direct snapshotting. Additionally the current config remains on 1.7 and we need a 1.8 setup. Using that same approach (for the CI instance) I created an ubuntu instance for 1.8 a few weeks ago and I've continued to have issues with that setup with the intel and clang compiler so the level of effort here seems to be equivalent anyways.

I think the right path here is to create and document a clean 1.8 install for multiple environments and ensure that they work as expected. From that config we can create or update the CI instance.

[eap]

Another issue related to all of this, but likely deserving its own discussion is clang support. Currently clang has substantial behavior differences between older 13 and 14 versions and more recent 17-19 versions. The version you get depends on the OS base with debian based distributions sending out older builds and redhat based distributions including the recent versions.

@stiggy87 has spend more time on this than I have but we both agree that its a mess and is making continued support quite difficult. For the moment we are likely to skip clang support in the AMI (rocky base) and I'd advocating a temporary pause on clang support until the OS platforms agree on a version, or until we can determine which version to support with sound justification for the invested effort.

[stiggy87]

As @eap has mentioned, Clang has been a pain, and I agree that we should pause it. I can go over all the roadblocks and workarounds I've tried to get passed compilation failures, but that's for another time.

So far, I've gotten GNU and Intel (icc) in an instance working, and I'm writing the documentation for that right now.

[srherbener]

I like @eap's proposal to go with gcc, intel now, and add in clang later. JCSDA is quite interested in exploring AWS instances as a development platform and this proposal will enable that sooner rather than later.

I think it would be good to have clang support in the long term, but it sounds like it would be easier to add this in when the version mess gets more manageable.