How do we build GFTL on an airgapped system

GeorgeVandenberghe-NOAA commented 8 months ago

The GFTL repository itself can be copied to an airgapped system with .. on connected system

git clone https://github.com/Goddard-Fortran-Ecosystem/gFTL-shared

then copy the gFTL-shared directory to the airgapped system and attempt further build. Upon doing so ON THE AIRGAPPED SYSTEM though the build generates

cd gFTL-shared/

rm -rf build
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/mnt/lfs4/HFIP/hfv3gfs/gwv/simt/simstack/netcdf140.492.460.mapl235.fms2301.crtm240.z .. -- The Fortran compiler identification is Intel 18.0.5.20180823 -- Detecting Fortran compiler ABI info -- Detecting Fortran compiler ABI info - done -- Check for working Fortran compiler: /apps/intel/compilers_and_libraries_2018.5.274/linux/mpi/intel64/bin/mpiifort - skipped -- Performing Test _LOGICAL_DEFAULT_KIND: SUCCESS (value=4) -- Performing Test _INT_DEFAULT_KIND: SUCCESS (value=4) -- Performing Test _ISO_INT8: SUCCESS (value=1) -- Performing Test _ISO_INT16: SUCCESS (value=2) -- Performing Test _ISO_INT32: SUCCESS (value=4) -- Performing Test _ISO_INT64: SUCCESS (value=8) -- Performing Test _REAL_DEFAULT_KIND: SUCCESS (value=4) -- Performing Test _DOUBLE_DEFAULT_KIND: SUCCESS (value=8) -- Performing Test _ISO_REAL32: SUCCESS (value=4) -- Performing Test _ISO_REAL64: SUCCESS (value=8) -- Performing Test _ISO_REAL128: SUCCESS (value=16) Cloning into '/mnt/lfs4/HFIP/hfv3gfs/gwv/simt/simstack/build/gFTL-shared/extern/gFTL'... fatal: unable to access 'https://github.com/Goddard-Fortran-Ecosystem/gFTL.git/': Failed connect to github.com:443; Connection timed out fatal: clone of 'https://github.com/Goddard-Fortran-Ecosystem/gFTL.git' into submodule path '/mnt/lfs4/HFIP/hfv3gfs/gwv/simt/simstack/build/gFTL-shared/extern/gFTL' failed Failed to clone 'extern/gFTL'. Retry scheduled Cloning into '/mnt/lfs4/HFIP/hfv3gfs/gwv/simt/simstack/build/gFTL-shared/extern/gFTL'...

How do we do this build on an airgapped system (NOAA jet tjet node in this example)?

tclune commented 6 months ago

So sorry - this issue apparently got buried during my winter vacation ...

This has come up before with a Cray system. The intent was that CMake should only reach outside if it cannot find the submodules, but the existing logic fails there. Adding in @mathomp4 so we can discusse.

Quite likely not a single gFTL/GFE user is relying on this capability, but without being certain, I'm reluctant to delete it. OTOH, we should be able to create a nice message telling the person to get the submodules ...

mathomp4 commented 6 months ago

@GeorgeVandenberghe-NOAA I can think of a few things.

Previously built gFTL

If you've already installed gFTL somewhere, then you'd just need to add the path to CMAKE_PREFIX_PATH when you run CMake.

Starting with gFTL-shared

If you are starting with gFTL-shared, then you'd want to git clone --recurse-submodules at which point I believe it wouldn't have a need to go out. For example, on a discover login node (with internet access), I did:

git clone --recurse-submodules git@github.com:Goddard-Fortran-Ecosystem/gFTL-shared.git

and then on a compute node (no internet):

cmake -B build -S .
cmake --build build --target install

worked just fine so it didn't try and fetch it.

Using GFE

Finally, I mainly install GFE libraries via GFE itself which always presents a "consistent" set of versions. For that:

git clone git@github.com:Goddard-Fortran-Ecosystem/GFE.git
cd GFE
git submodule update --init

and then build as usual.

GeorgeVandenberghe-NOAA commented 6 months ago

Thanks. I will modify my builds to try that.

On Mon, Feb 12, 2024 at 9:58 AM Matthew Thompson @.***> wrote:

@GeorgeVandenberghe-NOAA https://github.com/GeorgeVandenberghe-NOAA I can think of a few things.

If you've already installed gFTL somewhere, then you'd just need to add the path to CMAKE_PREFIX_PATH when you run CMake.

If you are starting with gFTL-shared, then you'd want to git clone --recurse-submodules at which point I believe it wouldn't have a need to go out. For example, on a discover login node (with internet access), I did:

git clone --recurse-submodules @.***:Goddard-Fortran-Ecosystem/gFTL-shared.git

and then on a compute node (no internet):

cmake -B build -S . cmake --build build --target install

worked just fine so it didn't try and fetch it.

Finally, I mainly install GFE libraries via GFE itself https://github.com/Goddard-Fortran-Ecosystem/GFE which always presents a "consistent" set of versions. For that:

git clone @.***:Goddard-Fortran-Ecosystem/GFE.git cd GFE git submodule update --init

and then build as usual.

— Reply to this email directly, view it on GitHub https://github.com/Goddard-Fortran-Ecosystem/gFTL/issues/209#issuecomment-1938837253, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDS4FW23HIHFZ6N427VUQLYTIUXNAVCNFSM6AAAAABA3URL6CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZYHAZTOMRVGM . You are receiving this because you were mentioned.Message ID: @.***>

--

George W Vandenberghe

Lynker Technologies at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2141

College Park, MD 20740

@.***

301-683-3769(work) 3017751547(cell)

Goddard-Fortran-Ecosystem / gFTL