[INSTALL]: CRTM 3.1.1 for emc

Hang-Lei-NOAA commented 4 months ago

Package name

CRTM

Package version/tag

release/REL-3.1.0

Build options

None, fix file included

Installation timeframe

Please add it into spack and install on RDHPCS machines

Other information

No response

WCOSS2

[X] Check this box if and only if your package should be installed on WCOSS2 Cactus and Dogwood (all spack-stack packages will be installed on Acorn). If not, you may disregard the rest of the items below and submit this request.

WCOSS2: General questions

No response

WCOSS2: Installation and testing

No response

WCOSS2: Technical & security review list

[ ] The code is mature, stable, and production ready
[ ] The code is does not and cannot use the internet, and does not contain URLs (http, https, ftp, etc.) except in comments
[ ] The package does not contain prebuilt binary files that have not been approved by NCO security review
[ ] The code has no publicly disclosed cybersecurity vulnerabilities and exposures (search https://cve.mitre.org/cve/)
[ ] The code is not prohibited by DHS, DOC, NOAA, or NWS
[ ] The code comes from a trusted source. Trusted sources include other NWS, NOAA, or DOC, agencies, or other Federal agencies that operate at a FISMA high or equivalent level. Additionally, trusted sources could be third-party agencies through which there is an existing SLA on file (such as RedHat).
[ ] The code is actively maintained and supported (it continues to get updates, patches, etc.)
[ ] The code is not maintained by a private entity operating in a foreign country (if it is, make a note below)
[ ] There is sufficient documentation to support maintenance
[ ] There are no known security vulnerabilities or weaknesses
[ ] Installing and running the code does not require privileged processes/users
[ ] There are no software dependencies that are unapproved or have security concerns (if there are, make a note below)
[ ] There are no concerns related to SA, SI, and SC NIST control families

WCOSS2: Additional comments

No response

climbfuji commented 4 months ago

@BenjaminTJohnson A request was made by EMC to add CRTM 3.1.0 to spack-stack. We'll do this as part of spack-stack-1.8.0, which we hope to release early September. Which version/tag should we use? Thanks!

AlexanderRichert-NOAA commented 4 months ago

@BenjaminTJohnson @Hang-Lei-NOAA is this a release/tag that already exists? Can you confirm repo URL and commit hash?

BenjaminTJohnson commented 4 months ago

https://github.com/JCSDA/CRTMv3/releases/tag/v3.1.0-skylabv8

262599de6f191e0bec460c6f9fbad8b2c3ac732c

I'm happy to create a new tag if requested.

Hang-Lei-NOAA commented 4 months ago

@Dom this requires the special version for EMC as the name included in the ticket. Please note that weather operations uses the specific version in the crtm repository.

On Tue, Jul 23, 2024 at 1:12 PM Benjamin Johnson @.***> wrote:

https://github.com/JCSDA/CRTMv3/releases/tag/v3.1.0-skylabv8

262599de6f191e0bec460c6f9fbad8b2c3ac732c

I'm happy to create a new tag if requested.

— Reply to this email directly, view it on GitHub https://github.com/JCSDA/spack-stack/issues/1185#issuecomment-2245797143, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFHAZZWVYKQYIIOTL53ZN2FHTAVCNFSM6AAAAABKSA6DPWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBVG44TOMJUGM . You are receiving this because you were mentioned.Message ID: @.***>

BenjaminTJohnson commented 4 months ago

@Hang-Lei-NOAA there are no special EMC versions with CRTM v3.x like we did with CRTM 2.x.y. Everyone gets the same version. I have taken care to ensure that the appropriate binary files for EMC requirements are present.

edit: happy to make a special tag if necessary, but it will be the same code / binary data.

Hang-Lei-NOAA commented 4 months ago

@BenjaminTJohnson correct The code won’t change. what I mean is that the EMC usually uses different fix files from the main release. Installing from the main release will not work for gfs model. This mistake has happened in crtm 2.x installation. Therefore, the spack-stack needs to notice the difference in tag name.

BenjaminTJohnson commented 4 months ago

Got it. Just to be clear, for v3.x, the calling models will have to conform to the CRTMv3 structure and binary files. The binary file naming convention and content has been confirmed to be consistent with the EMC/NCO expectation, to the best of my knowledge. However, in module format, the structure of the source code shouldn't be an issue at all. Some work will need to be done on your side to ensure compatibility. I'll be as responsive as possible during the process to avoid any delays.

climbfuji commented 3 months ago

GSI is not ready for CRTMv3 yet

climbfuji commented 3 months ago

@BenjaminTJohnson Are you ok with adding crtm@3.1.? to spack-stack-1.8.0 for extended testing by EMC? Otherwise, we'll have to add it after the fact in an addon environment (significantly more work). Thanks!

BenjaminTJohnson commented 3 months ago

@climbfuji they're testing v3.1.1 currently:

https://github.com/JCSDA/CRTMv3/issues/167

I think I'd aim for that instead. When do you plan to start this? I have a couple of changes that need to go into the release/REL-3.1.1 branch related to the above issue, and plan to issue a new tag either today or tomorrow.

What do you think? They're not even going to use 3.1.0 because some key things they need were missing.

climbfuji commented 3 months ago

We'll cut the release branch on Friday

BenjaminRuston commented 3 months ago

@climbfuji thanks for pointing me to the issue

climbfuji commented 3 months ago

Based on the discussion in https://github.com/JCSDA/CRTMv3/pull/175, we'll defer this to a post-install step after the spack-stack-1.8.0 release. @BenjaminTJohnson, myself and possibly one or two others will meet in the weeks following the spack-stack release to finalize the CRTMv3 set up in spack.

RussTreadon-NOAA commented 1 week ago

What is the status of this issue? Do we have an estimated completion date?

JCSDA / spack-stack