search

NOAA-GSL / ExascaleWorkflowSandbox

Other
2 stars 2 forks source link

Make install scripts portable #33

Closed christopherwharrop-noaa closed 1 year ago

christopherwharrop-noaa commented 1 year ago

The existing install scripts currently in use for building Docker containers are not portable because they assume that gcc 9.4.0 is already installed on the system. If that is not the case, the install will fail. This PR introduces two more installation scripts to help make the install process more portable. When the required compiler, gcc 9.4.0, is not available, a bootstrap compiler must be built and then used to install gcc 9.4.0 before installing Flux and Parsl. A install_bootstrap.sh install script is added to build and install the bootstrap compiler if one is needed. This script builds and installs gcc 9.4.0 with the system default compiler. However, in order to make Spack build cache mirrors work correctly on potential destination machines with no internet access, we must remove the dependency on the default compiler. This is done with install_base.sh. The install_base.sh script then builds gcc 9.4.0 again using the gcc 9.4.0 compiler built by the bootstrap compiler. This results in a clean version of gcc 9.4.0 installed without any dependencies on libraries built with the default system compiler. At this point, Parsl and Flux may be installed with no issues, and the environment is suitable for creation of Spack build cache that can be used on other systems.

The current containers used for CI testing already have gcc 9.4.0 as the default compiler, so when install_bootstrap.sh and install_base.sh are installed in CI, there is no change in behavior, and the installations of the bootstrap compiler and gcc 9.4.0 are skipped. However, when installing on other HPC systems, the same install scripts that are used in CI will now correctly install the environment without modification.

christopherwharrop-noaa commented 1 year ago

@NaureenBharwaniNOAA - Please take a look when you have time.