search

EESSI / compatibility-layer

Compatibility layer of the EESSI project
https://eessi.github.io/docs/compatibility_layer
GNU General Public License v2.0
11 stars 21 forks source link

final tarball for built compat layer not created correctly by bot/build.sh #189

Open boegel opened 1 year ago

boegel commented 1 year ago

As shown in #188, building the 2023.06 compat layer autonomously worked, but creating the tarball for ingestion failed because of a bug at the end of the bot/build.sh script.

Relevant part of output of the Slurm build job:

PLAY RECAP *********************************************************************
localhost                  : ok=63   changed=38   unreachable=0    failed=0    skipped=3    rescued=0    ignored=1

Left container; tar'ing up /tmp/bot/EESSI/eessi.BM69QGLjmn for future inspection
...
created tarball '/mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.06/pr_188/event_bf958ee0-0a82-11ee-93ec-8e0bfaa03c26/run_001/linux_aarch64_generic/eessi-2023.06-compat/job_4940_1686744594.tgz'
To resume work add '--resume /tmp/bot/EESSI/eessi.BM69QGLjmn'
Directory //cvmfs/pilot.eessi-hpc.org/versions/2023.06 was not created, not creating tarball.

The job_*.tgz mentioned not only includes /cvmfs/..., but also /tmp and apptainer_cache, so it can't be used for ingestion.

trz42 commented 1 year ago

Left container; tar'ing up /tmp/bot/EESSI/eessi.BM69QGLjmn for future inspection ... created tarball '/mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.06/pr_188/event_bf958ee0-0a82-11ee-93ec-8e0bfaa03c26/run_001/linux_aarch64_generic/eessi-2023.06-compat/job_4940_1686744594.tgz' To resume work add '--resume /tmp/bot/EESSI/eessi.BM69QGLjmn' Directory //cvmfs/pilot.eessi-hpc.org/versions/2023.06 was not created, not creating tarball.



The `job_*.tgz` mentioned not only includes `/cvmfs/...`, but also `/tmp` and `apptainer_cache`, so it can't be used for ingestion.

As the first line states, this tarball is intended for future inspection not for ingestion. It's essentially the complete temporary directory used during the job or more precisely what install_compatibility_layer.sh uses as base folder for temporary data. It includes cvmfs, home, tmp and possibly container_cachedir.

What's missing in the bot/build.sh script is assigning a value to eessi_tmpdir. Adding a line such as

eessi_tmpdir=${STORAGE}

before https://github.com/EESSI/compatibility-layer/blob/114f12ab6806232ea7b894ac5af73aeb7d2443fa/bot/build.sh#L97-L106 would probably fix it.