fail-to-set-ambient-capability in build post on CentOS7 with old centosplus kernel

[DrOllyGomez]

Version of Singularity What version of Singularity are you using?

singularity-ce version 3.10.0-1.el7

Describe the bug A clear and concise description of what the bug is.

Regardless of whether I use the yum bootstrap agent or the debootstrap agent, whilst -I am able to install base systems eg CentOS7, Ubuntu16, Ubuntu20 -I am unable to install any further software at the %post scriptlet stage

Running as root, I make the call to singularity build with a def file, (using either of the two agents named above), and I get output like this:

-many I:configuring things etc- I: Configuring libapt-pkg6.0:amd64... I: Configuring apt... I: Configuring libc-bin... I: Base system installed successfully. INFO: Running post scriptlet ERROR : Failed to set ambient capability: Invalid argument FATAL: While performing build: while running engine: exit status 1

I reiterate that I can make .sif images successfully with base systems, I can shell into them and look around them.

To Reproduce Steps to reproduce the behavior:

This minimal def file creates the error.... Bootstrap: debootstrap OSVersion: focal MirrorURL: http://mirror.bytemark.co.uk/ubuntu/ %post apt-get update && apt-get install -y wget

The same file without the %post section label and scriptlet do not produce the error, and produce a working sif container.

Expected behavior

A .sif will be created, that I can shell into, and use the wget program

OS / Linux Distribution Which Linux distribution are you using?

NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7"

Installation Method Write here how you installed SingularityCE. Eg. RPM, source.

I used:

wget https://github.com/sylabs/singularity/releases/download/v3.10.0/singularity-ce-3.10.0-1.el7.x86_64.rpm

then

yum install ./singularity-ce-3.10.0-1.el7.x86_64.rpm

I will attached a text file that contains the output to the console of the install process

Additional context 1.I read in... https://sylabs.io/guides/latest/user-guide/appendix.html#debootstrap-build-agent .. that on a Centos system, to use the debootstrap agent, its necessary to install deboostrap on the build system with: sudo yum update && sudo yum install epel-release && sudo yum install debootstrap.noarch ...so I have done that, FWIW. I reiterate that the problem seems to be generic, it happens with the yum agent also.

2.FWIW I have seen a comment: W: Cannot check Release signature; keyring file not available /usr/share/keyring s/ubuntu-archive-keyring.gpg

Many thanks in advance for a fix, any help, comments, and any requests on how to diagnose the problem further. install.txt

[dtrudg]

Hi @DrOllyGomez - what kernel version do you have installed on this host? i.e. output of uname -a

Also, please could you provide debug output from the build. I.E. everything you get from singularity --debug build .... (adding the --debug flag to your build command).

[DrOllyGomez]

Hiya. Thanks for the reply.

$uname -a 3.10.0-327.36.3.el7.centos.plus.x86_64 #1 SMP Tue Oct 25 03:09:35 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

attaching output of debug build... debugout.txt

[dtrudg]

3.10.0-327.36.3.el7.centos.plus.x86_64 https://github.com/sylabs/singularity/pull/1 SMP Tue Oct 25 03:09:35 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

This is an extremely old (Oct 2016) kernel build, and not one we would be testing against. Note also that it is a centos plus kernel, where centos plus states...

These packages are not part of the upstream distribution and extend CentOS's functionality at the expense of upstream compatibility.

Are you able to verify against the current CentOS 7 stock kernel, which is:

kernel-3.10.0-1160.66.1.el7.x86_64

[DrOllyGomez]

ok, understood the point about the kernel. I will see if I can change, but not sure when.

Are you able to verify against the current CentOS 7 stock kernel,

as above, probably, but maybe not soon. Probably best to close this ticket? Thanks for your comments.

[dtrudg]

Let's keep the ticket open - if I have an hour to try and replicate this week, I will take a look.

[dtrudg]

Attempting a reproducer on a RHEL7 install that isn't quite current, but has a Nov 2021 kernel...

06:02 PM $ singularity --version
singularity-ce version 3.10.0-1.el7

06:02 PM $ cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.9 (Maipo)

06:02 PM $ uname -a
Linux dev-rhel7 3.10.0-1160.49.1.el7.x86_64 #1 SMP Tue Nov 9 16:09:48 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

06:04 PM $ cat debootstrap.def 
Bootstrap: debootstrap
OSVersion: focal
MirrorURL: http://mirror.bytemark.co.uk/ubuntu/

%post
   apt-get update && apt-get install -y wget

$ sudo singularity build debootstrap.sif debootstrap.def 
INFO:    Starting build...
W: Cannot check Release signature; keyring file not available /usr/share/keyrings/ubuntu-archive-keyring.gpg
...
I: Base system installed successfully.
INFO:    Running post scriptlet
+ apt-get update
...
done.
INFO:    Creating SIF file...
INFO:    Build complete: debootstrap.sif

As seen above, it completes without issue.

Without having had any other reports, I believe this is going to be due to the very old centosplus kernel, and I'm afraid putting in a workaround for a kernel that is outdated and subject to security issues isn't something we're likely to do. If you do see it with a somewhat recent kernel, then we can look into it further. Thanks.

[DrOllyGomez]

Understood. Many thanks for demo and info. Best. Mike

---

From: Dave Trudgian @.> Sent: 07 June 2022 23:10 To: sylabs/singularity @.> Cc: Mike Harbour @.>; Mention @.> Subject: Re: [sylabs/singularity] Use of post scriptlets causes fail-to-set-ambient-capability build crashes on CentOS7 host for debootstrap and yum agents/modules (Issue #844)

Attempting a reproducer on a RHEL7 install that isn't quite current, but has a Nov 2021 kernel...

06:02 PM $ singularity --version singularity-ce version 3.10.0-1.el7

06:02 PM $ cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.9 (Maipo)

06:02 PM $ uname -a Linux dev-rhel7 3.10.0-1160.49.1.el7.x86_64 #1 SMP Tue Nov 9 16:09:48 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

06:04 PM $ cat debootstrap.def Bootstrap: debootstrap OSVersion: focal MirrorURL: http://mirror.bytemark.co.uk/ubuntu/http://mirror.bytemark.co.uk/ubuntu/

%post apt-get update && apt-get install -y wget

$ sudo singularity build debootstrap.sif debootstrap.def INFO: Starting build... W: Cannot check Release signature; keyring file not available /usr/share/keyrings/ubuntu-archive-keyring.gpg ... I: Base system installed successfully. INFO: Running post scriptlet

• apt-get update ... done. INFO: Creating SIF file... INFO: Build complete: debootstrap.sif

As seen above, it completes without issue.

Without having had any other reports, I believe this is going to be due to the very old centosplus kernel, and I'm afraid putting in a workaround for a kernel that is outdated and subject to security issues isn't something we're likely to do.

— Reply to this email directly, view it on GitHubhttps://github.com/sylabs/singularity/issues/844#issuecomment-1149223131, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AL5UYOK74N74SY27CJ7CA2TVN7CDRANCNFSM5X47ZFIA. You are receiving this because you were mentioned.Message ID: @.***>

This e-mail including any attachments is for the intended recipient(s) only and contains information or material, which may be confidential and/or legally privileged. If you are not the intended recipient, any disclosure, copying, use, distribution or retaining of the information or material contained in this e-mail is unauthorised and prohibited. Please notify the sender immediately by e-mail if you have received it by mistake and delete the e-mail immediately from your system. We have taken steps to ensure that this email (and any attachments) are free from computer viruses and the like. However, it is your responsibility to ensure that it is actually virus free. Any emails that you send to us may be monitored for the purposes of ascertaining whether the communication complies with the law and our policies. Storm Therapeutics Limited. Registered Office: c/o Moneta Building, Babraham Research Campus, Cambridge, England, CB22 3AT. Registered in England under number 09553473.

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast, a leader in email security and cyber resilience. Mimecast integrates email defenses with brand protection, security awareness training, web security, compliance and other essential capabilities. Mimecast helps protect large and small organizations from malicious activity, human error and technology failure; and to lead the movement toward building a more resilient world. To find out more, visit our website.