Please describe the package or library you would like to add to hpc-stack.
The Scotch distribution is a set of programs and libraries which implement the static mapping and sparse matrix reordering algorithms developed within the Scotch project. We would like to utilize the graph partitioning capability of Scotch in the WW3 model. Here is the link to the Scotch repository:
https://gitlab.inria.fr/scotch/scotch
What applications at NOAA will be using this package or library?
UFS-WEATHER-MODEL:
GFSv17
GEFSv13
UFS-COASTAL
GLWUv3
Is there already a package or library in hpc-stack that provides this, or related, functionality?
NA
Additional context
I have tested the compilation of SCOTCH with GNU and Intel compiler on both Hera and Orion. The compilation with intel is a bit tricky, so here I added the step-by-step instruction:
Hera
cd scotch
module purge
module load cmake/3.20.1
module use /scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/modulefiles/stack
module load hpc/1.2.0
module load hpc-intel/2022.1.2
module load hpc-impi/2022.1.2
module load netcdf/4.9.0
module load gnu
mkdir build
cd build
cmake VERBOSE=1 -DCMAKE_Fortran_COMPILER=ifort -DCMAKE_C_COMPILER=icc -DCMAKE_INSTALL_PREFIX=<path to scotch>/install -DTHREADS=OFF -DCMAKE_BUILD_TYPE=Release .. | & tee cmake.out-rr
make VERBOSE=1 | & tee make.out-rr
make install
make scotch
make ptscotch
I compiled scotch following this instruction on Hera in
/scratch2/COASTAL/coastal/save/Ali.Abdolali/hpc-stack/scotch/install
Orion
cd scotch
module purge
mkdir -p $HOME/modulefiles/gcc
cp /apps/modulefiles/core/gcc/10.2.0 $HOME/modulefiles/gcc/
Edit the file $HOME/modulefiles/gcc/10.2.0:
Comment out the 'family "compiler"' line
Comment out the 'prepend-path MODULEPATH' line
module use $HOME/modulefiles
module load gcc/10.2.0
module load cmake/3.22.1
module use /apps/contrib/NCEP/libs/hpc-stack/modulefiles/stack
module load hpc/1.1.0
module load hpc-intel/2018.4
module load hpc-impi/2018.4
module load netcdf/4.7.4
module load jasper/2.0.25
module load zlib/1.2.11
module load png/1.6.35
module load hdf5/1.10.6
module load bacio/2.4.1
module load g2/3.4.2
module load w3emc/2.9.2
module load esmf/8_2_0
mkdir build
cd build
cmake VERBOSE=1 -DCMAKE_Fortran_COMPILER=ifort -DCMAKE_C_COMPILER=icc -DCMAKE_INSTALL_PREFIX=<path to scotch>/install -DTHREADS=OFF -DCMAKE_BUILD_TYPE=Release .. | & tee cmake.out-rr
make VERBOSE=1 | & tee make.out-rr
make install
make scotch
make ptscotch
I compiled scotch following this instruction on Orion in
/work/noaa/marine/ali.abdolali/Source/hpc-stack/scotch/install
The Intel compiler depends on GNU for some of its functionality, and needs a more recent version of GNU compiler to function correctly. The default version of GNU that comes with most distribution is pretty old.
So if WCOSS2 has a more recent version as the default then it should simply work. Based on the GNU module loaded on hera, it seems to work fine with:
gcc (Spack GCC) 9.2.0
If WCOSS2 had an older version of GNU then you see if there is a module for GNU with a more recent compiler.
Will This Package be Needed in an Operational Application?
Yes
WCOSS System Software Request Checklist
General questions:
Supervisor or sponsor of the requester: @AvichalMehra-NOAA
The license of the package is approved by NCO.
Is it provided by a trusted source? Trusted sources include other NWS, NOAA, or DOC, agencies, or other Federal agencies that operate at a FISMA high or equivalent level. Additionally, trusted sources could be third-party agencies through which there is an existing SLA on file (such as RedHat).
Is software support offered (is it being updated and patched). Yes, the main developers agreed to support the software.
If maintained by a private entity, does the entity operate in a foreign country, especially a prohibited foreign country (China, Russia, Iran, North Korea, etc.).
Is there sufficient documentation to support maintenance Yes
Are there known vulnerabilities or weaknesses No
Is there a need for privileged processes
Are there software dependencies, are those dependencies approved or do they have any security concerns
Are there any other concerns related to SA, SI, and SC control families
Please describe the package or library you would like to add to hpc-stack. The Scotch distribution is a set of programs and libraries which implement the static mapping and sparse matrix reordering algorithms developed within the Scotch project. We would like to utilize the graph partitioning capability of Scotch in the WW3 model. Here is the link to the Scotch repository: https://gitlab.inria.fr/scotch/scotch
What applications at NOAA will be using this package or library? UFS-WEATHER-MODEL:
Is there already a package or library in hpc-stack that provides this, or related, functionality? NA
Additional context I have tested the compilation of SCOTCH with GNU and Intel compiler on both Hera and Orion. The compilation with intel is a bit tricky, so here I added the step-by-step instruction: Hera
I compiled scotch following this instruction on Hera in
/scratch2/COASTAL/coastal/save/Ali.Abdolali/hpc-stack/scotch/installOrion
I compiled scotch following this instruction on Orion in
/work/noaa/marine/ali.abdolali/Source/hpc-stack/scotch/installThe Intel compiler depends on GNU for some of its functionality, and needs a more recent version of GNU compiler to function correctly. The default version of GNU that comes with most distribution is pretty old. So if WCOSS2 has a more recent version as the default then it should simply work. Based on the GNU module loaded on hera, it seems to work fine with:
gcc (Spack GCC) 9.2.0If WCOSS2 had an older version of GNU then you see if there is a module for GNU with a more recent compiler.Will This Package be Needed in an Operational Application? Yes
WCOSS System Software Request Checklist
General questions:
Supervisor or sponsor of the requester: @AvichalMehra-NOAA The license of the package is approved by NCO.
Software name and version, specific URL to the software SCOTCH https://gitlab.inria.fr/scotch/scotch
Software type - New or Upgrade existing? New
Justification ( List NPS models using the software)
Completion Time requested
Software License including all Dependency Software Licenses
Support contact(s) who must have a WCOSS account
Dependency Software list
Installation instructions
Test and verification instructions
Technical Review Checklist for open source software - Review the source code to answer the following questions
Licenses for the requested software and its dependencies
Licenses - confirm the software Licenses are acceptable
Maturity
Acceptable - Stable, production, or equivalent
Self-contained
No external http, https, ftp, or other URI exists except that in comments
No binary files in the package unless they are in the approved list
No publicly disclosed cybersecurity vulnerabilities and exposures
Searching https://cve.mitre.org/cve/
Security High Level Checklists
Is it prohibited by DHS/DOC/NOAA/NWS
Is it provided by a trusted source? Trusted sources include other NWS, NOAA, or DOC, agencies, or other Federal agencies that operate at a FISMA high or equivalent level. Additionally, trusted sources could be third-party agencies through which there is an existing SLA on file (such as RedHat).
Is software support offered (is it being updated and patched). Yes, the main developers agreed to support the software.
If maintained by a private entity, does the entity operate in a foreign country, especially a prohibited foreign country (China, Russia, Iran, North Korea, etc.).
Is there sufficient documentation to support maintenance Yes
Are there known vulnerabilities or weaknesses No
Is there a need for privileged processes
Are there software dependencies, are those dependencies approved or do they have any security concerns
Are there any other concerns related to SA, SI, and SC control families