Updates for EESSI compat layer 2022.11

[trz42]

This includes a number of updates to the current version (2021.12):

File bootscript-prefix.sh:

• The script bootstrap-prefix.sh has been sync'ed with updates to the upstream version (https://github.com/gentoo/prefix.git) until a recent update on Nov 2 2022. The script here still includes a few changes we may want to revisit i.e. if they are necessary or if they can be removed to allow us to just use the upstream script.
• Changes that are still present (diff gentoo-prefix/scripts/bootstrap-prefix.sh EESSI-compatibility-layer/bootstrap-prefix.sh):

  590c590,593
  <       local theshell=${SHELL##*/}
  ---
  >       # from gentoo/prefix
  >       #local theshell=${SHELL##*/}
  >       # from EESSI/compatibility-layer
  >       local theshell=$(echo "${SHELL##*/}" | cut -f1 -d ' ')
  2267,2268c2270,2271
  <       DISTFILES_G_O="http://distfiles.prefix.bitzolder.nl"
  <       DISTFILES_PFX="http://distfiles.prefix.bitzolder.nl/prefix"
  ---
  >       DISTFILES_G_O="https://distfiles.prefix.bitzolder.nl"
  >       DISTFILES_PFX="https://distfiles.prefix.bitzolder.nl/prefix"
  2825c2828
  <                       [[ ${TODO} == 'noninteractive' ]] && ans=yes ||
  ---
  >                       [[ ${TODO} == 'noninteractive' ]] && ans=no ||

File ansible/playbooks/roles/compatibility_layer/defaults/main.yml:

• The version has been bumped to 2022.11. (eessi_version: "2022.11")
• The gentoo overlay currently uses a fork (https://github.com/trz42/gentoo-overlay.git). This can be reverted after https://github.com/EESSI/gentoo-overlay/pull/84 has been merged. (custom_overlays.url: https://github.com/trz42/gentoo-overlay.git)
• The gentoo commit (see gentoo_git_commit: cec3214ef5d5661e28c9d2c5b5750b27c27c5435) being used is updated to a more recent version (from Nov 3 2022). A bit information about history of used commits was added. Can be removed if deemed unnecessary.
• The default gcc version has been increased to 10.4.0 (see prefix_default_gcc: 10.4.0). (Requirement in upstream bootstrap-prefix.sh script.)
• However, the version of gcc being used had to be restricted to anything before 10.4.1 because some 10.4.1_p* ebuilds were added to https://github.com/gentoo/gentoo/sys-devel/gcc recently. While those installed fine during the bootstrap stages 2/3, it was not immediately clear how to set them as default with gcc-config. Simply setting them via prefix_default_gcc (see item on default gcc) didn't work. (see line >=sys-devel/gcc-10.4.1 for prefix_mask_packages:)
• The setting for prefix_singularity_command were changed to unset LD_LIBRARY_PATH (the bootstrap-prefix.sh in 2021.12 had a slight spelling typo in the check if LD_LIBRARY_PATH is set, hence went on even if it was set). The updated bootstrap-prefix.sh checks for the correct variable.

File ansible/playbooks/roles/compatibility_layer/tasks/install_packages.yml:

• Task to change file ownership needed root permissions. (Could be an issue with testing setup. Might be good to verify if it is needed in other environments.)

[amadio]

The default gcc version has been increased to 10.4.0 (see prefix_default_gcc: 10.4.0). (Requirement in upstream bootstrap-prefix.sh script.)

You know you don't need to stick to that after installing, right? You can and should update to a newer version of GCC if you want. Only the bootstrap process is a bit finicky, but once you bootstrap, you are usually free from the constraints. You can also install multiple versions of GCC (i.e. GCC 12 for CPU, GCC 11 for CUDA), which you can use later as gcc-$version on the command line even if they are not the default.

[trz42]

The default gcc version has been increased to 10.4.0 (see prefix_default_gcc: 10.4.0). (Requirement in upstream bootstrap-prefix.sh script.)

You know you don't need to stick to that after installing, right? You can and should update to a newer version of GCC if you want. Only the bootstrap process is a bit finicky, but once you bootstrap, you are usually free from the constraints. You can also install multiple versions of GCC (i.e. GCC 12 for CPU, GCC 11 for CUDA), which you can use later as gcc-$version on the command line even if they are not the default.

Good to know. I think for EESSI we only need one version that is then used to build up tools chains (incl compilers) in the software layer. In the past we ran into issues with too recent versions of GCC in that step. Hence, we'd rather like to restrict ourselves to the oldest version of GCC in the compat layer. Might change in the future when we move to more recent tool chains in the software layer.

[amadio]

I see. I suggest to try using the toolchain from the compat layer to build the software layer, to avoid having to add ld wrappers, etc, since the toolchain is already configured to work well with the non-standard glibc and linker. It will also let you control binutils with eselect-binutils, etc. I read the diffs and the reason to use <GCC-11 is because GCC-11 needs C++11 support to build, which may not be available on old OSs like CentOS 7, etc. Other than that, you should be able to remove the restriction, as I believe other compilation problems have been solved.

[trz42]

I'd rather reconfigure the ansible-lint check to ignore the Jinja template issue. The script has run fine a dozen of times. The "ERROR" only states that templates should be at the end, so it doesn't matter. Also curious why we need to fix this here. Maybe it is better to do a code quality hackathon than imposing unrelated code improvements to PRs.

[trz42]

Probably interesting finds on building the compat layer:

• Wasn't able to build direnv easily, so removed it from the overlay sets. nss and stop built without any issues. rpm wasn't added as it didn't seem needed for GPU support.
• Building a compat layer from scratch was failing with the latest gentoo commits (Nov 05, Nov 03), only after going back to a commit from Oct 28 it again worked ... on x86_64.
• On aarch64 I ran into issues that community.general.portage is not a known ansible module action. So, will have to revert that until a proper fix is found. Previously, only portage was used but ansible-lint complained something like

  fqcn[action]: Use FQCN for module actions, such `<namespace>.<collection>.portage`. (warning)
  [1281](https://github.com/EESSI/compatibility-layer/actions/runs/3385280194/jobs/5623244261#step:4:1282)
  ansible/playbooks/roles/compatibility_layer/tasks/add_overlay.yml:22 Action `portage` is not FQCN.
  [1282](https://github.com/EESSI/compatibility-layer/actions/runs/3385280194/jobs/5623244261#step:4:1283)

• A bit curious what could be the issue: Are the containers we use to build the compat layer for x86_64 and aarch64 different wrt the ansible installation?

[trz42]

This PR has become obsolete with new developments in early 2023.