search

EESSI / compatibility-layer

Compatibility layer of the EESSI project
https://eessi.github.io/docs/compatibility_layer
GNU General Public License v2.0
11 stars 21 forks source link

add support for letting the bot build compat layer #163

Closed bedroge closed 1 year ago

bedroge commented 1 year ago

The new script install-compatibility-layer.sh will run Ansible inside an Apptainer container (generated with a modified version of the existing Dockerfile, updated to Rocky 8).

Haven't managed to do a full test run, due to issues with the bootstrapping process: it fails to fetch some source file, so I'll probably have to update the gentoo commit that we use.

bedroge commented 1 year ago

This seems to work now. I've tested it, but had to disable the package set installation (as, for instance, archspec wouldn't install due to changes in gentoo, unrelated to this PR) and use a more up-to-date version of the bootstrap script and gentoo commit.

bedroge commented 1 year ago

Initially, I switched from CentOS to Rocky for the container, but Rocky 8 only provides Docker containers for arm64 and amd64. So, in the end, I switched to Debian 11, which supports a whole range of architectures including ppc64le (and the unstable Debian image already supports riscv64 as well).

ocaisa commented 1 year ago

See https://github.com/EESSI/compatibility-layer/actions/runs/3903315061/jobs/6667515735#step:7:3652 for the failure...no rust compiler for POWER?

ocaisa commented 1 year ago

almalinux:8 does have a container for POWER

bedroge commented 1 year ago

almalinux:8 does have a container for POWER

Yeah, I considered that one as well, but since Debian is a bit more active with respect to riscv64 as well (which we probably want/need in the future) and the distro shouldn't really matter, I preferred it over Alma.

bedroge commented 1 year ago

See https://github.com/EESSI/compatibility-layer/actions/runs/3903315061/jobs/6667515735#step:7:3652 for the failure...no rust compiler for POWER?

Weird, I think it should exist: https://packages.debian.org/bullseye/rustc

bedroge commented 1 year ago

See https://github.com/EESSI/compatibility-layer/actions/runs/3903315061/jobs/6667515735#step:7:3652 for the failure...no rust compiler for POWER?

Weird, I think it should exist: https://packages.debian.org/bullseye/rustc

Fixed by adding rustc and libssl-dev to the container.

eessi-bot[bot] commented 1 year ago
New job on instance eessi-bot-citc-aws for architecture aarch64-graviton2 in job dir /mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.02/pr_163/3593 date job status comment
Feb 03 13:19:56 UTC 2023 submitted job id 3593 awaits release by job manager
Feb 03 13:20:31 UTC 2023 released job awaits launch by Slurm scheduler
Feb 03 13:21:33 UTC 2023 running job 3593 is running
Feb 03 13:23:37 UTC 2023 finished :cry: FAILURE
  • Found slurm output slurm-3593.out in job dir
  • Slurm output lacks message "No missing modules!".
  • Slurm output lacks message about created tarball.
  • No tarball matching eessi-*software-*.tar.gz found in job dir.
eessi-bot[bot] commented 1 year ago
New job on instance eessi-bot-citc-aws for architecture aarch64-graviton2 in job dir /mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.02/pr_163/3604 date job status comment
Feb 03 16:27:38 UTC 2023 submitted job id 3604 awaits release by job manager
Feb 03 16:27:54 UTC 2023 released job awaits launch by Slurm scheduler
Feb 03 16:28:57 UTC 2023 running job 3604 is running
Feb 03 16:34:04 UTC 2023 finished :cry: FAILURE
  • Found slurm output slurm-3604.out in job dir
  • Slurm output lacks message "No missing modules!".
  • Slurm output lacks message about created tarball.
  • No tarball matching eessi-*software-*.tar.gz found in job dir.
bedroge commented 1 year ago

Successfully ran this on x86_64:

TASK [compatibility_layer : Run ReFrame tests] *********************************
ok: [localhost]

RUNNING HANDLER [compatibility_layer : Generate locales] ***********************
changed: [localhost]

PLAY RECAP *********************************************************************
localhost                  : ok=57   changed=31   unreachable=0    failed=0    skipped=3    rescued=0    ignored=1
bedroge commented 1 year ago

This PR includes so many different things now, I'm splitting it up into smaller ones: