search

NVIDIA / enroot

A simple yet powerful tool to turn traditional container/OS images into unprivileged sandboxes.
Apache License 2.0
649 stars 95 forks source link

enroot+setcap doesn't on updates #79

Closed krono closed 3 years ago

krono commented 3 years ago

I'm running into caps issues with 3.3.0 on ppc64

excerpt:

[INFO] Converting whiteouts...

0% 0:9=0s 5ade347cf74df0ad62b82c57b0a6f0411ef9ecad9336a99c9e116ee5d1ff3d82
enroot-aufs2ovlfs: failed to set capabilities: Operation not permitted
#dnf list installed | grep enroot
enroot.ppc64le                                 3.3.0-1.el7
enroot+caps.ppc64le                            3.3.0-1.el7
 getcap /usr/bin/enroot-aufs2ovlfs
/usr/bin/enroot-aufs2ovlfs =
krono commented 3 years ago

Manually doing

setcap cap_sys_admin,cap_mknod+pe "$(command -v enroot-aufs2ovlfs)"
setcap cap_sys_admin+pe "$(command -v enroot-mksquashovlfs)"

helps tho

3XX0 commented 3 years ago

Weird, did it work at any point after you installed things for the first time? Also, does it work if you reinstall the +caps package?

krono commented 3 years ago

Yes, it worked with the 3.2.0 RPMs I produced from the specfile before merging ppc64. I filed the issue right after reinstalling the +caps package.

krono commented 3 years ago

I see that effect on two machines...

krono commented 3 years ago

Note that I dnf updated the packages… maybe a weird interplay of %preun and %post scripts?

3XX0 commented 3 years ago

Might be yeah, nothing changed with 3.3. I will try to see if I can repro with dnf, I usually test with yum

flx42 commented 3 years ago

I just got into a similar situation on Ubuntu, and for me the issue was that I previously did a make install of enroot from sources, so enroot+caps applied the setcap commands to those binaries (in /usr/local/bin) instead of the binaries installed through the enroot package.

Perhaps this is what happened to you too.

krono commented 3 years ago

I don't think so. I have 2 machines where that happened and one of them never saw the source code just the RPMs…

3XX0 commented 3 years ago

Any update on this one? I wasn't able to reproduce (even with dnf)

krono commented 3 years ago

Lets close then. I'll report when it happens again…

krono commented 3 years ago

Something is fishy in the kingdom of Denmark.

it is definitively running the scriplet, but to no avail.

Log of reinstall run ``` root@ic922-01 ~]# dnf reinstall --rpmverbosity=debug enroot+caps Updating Subscription Management repositories. Last metadata expiration check: 0:06:39 ago on Tue 21 Sep 2021 07:14:22 PM CEST. Dependencies resolved. ============================================================================================================================================================================================================ Package Architecture Version Repository Size ============================================================================================================================================================================================================ Reinstalling: enroot+caps ppc64le 3.3.1-1.el7 CONTAINER 3.3 k Transaction Summary ============================================================================================================================================================================================================ Total size: 3.3 k Installed size: 0 Is this ok [y/N]: y Downloading Packages: Running transaction check Transaction check succeeded. Running transaction test fdio: 8 reads, 3380 total bytes in 0.000011 secs fdio: 6 reads, 3264 total bytes in 0.000007 secs Transaction test succeeded. Running transaction fdio: 8 reads, 3380 total bytes in 0.000010 secs Preparing : 1/1 Reinstalling : enroot+caps-3.3.1-1.el7.ppc64le 1/2 D: closed db index /var/lib/rpm/Packages D: closed db index /var/lib/rpm/Enhancename D: closed db index /var/lib/rpm/Supplementname D: closed db index /var/lib/rpm/Suggestname D: closed db index /var/lib/rpm/Recommendname D: closed db index /var/lib/rpm/Transfiletriggername D: closed db index /var/lib/rpm/Filetriggername D: closed db index /var/lib/rpm/Sha1header D: closed db index /var/lib/rpm/Sigmd5 D: closed db index /var/lib/rpm/Installtid D: closed db index /var/lib/rpm/Dirnames D: closed db index /var/lib/rpm/Triggername D: closed db index /var/lib/rpm/Obsoletename D: closed db index /var/lib/rpm/Conflictname D: closed db index /var/lib/rpm/Providename D: closed db index /var/lib/rpm/Requirename D: closed db index /var/lib/rpm/Group D: closed db index /var/lib/rpm/Basenames D: closed db index /var/lib/rpm/Name D: closed db environment /var/lib/rpm D: opening db environment /var/lib/rpm cdb:0x401 D: opening db index /var/lib/rpm/Packages (none) mode=0x42 D: sanity checking 1 elements D: opening db index /var/lib/rpm/Name (none) mode=0x42 D: Plugin: calling hook tsm_pre in selinux plugin D: Plugin: calling hook tsm_pre in systemd_inhibit plugin D: System shutdown blocked (fd 66) D: running pre-transaction scripts D: computing 0 file fingerprints D: opening db index /var/lib/rpm/Basenames (none) mode=0x42 D: opening db index /var/lib/rpm/Group (none) mode=0x42 D: opening db index /var/lib/rpm/Requirename (none) mode=0x42 D: opening db index /var/lib/rpm/Providename (none) mode=0x42 D: opening db index /var/lib/rpm/Conflictname (none) mode=0x42 D: opening db index /var/lib/rpm/Obsoletename (none) mode=0x42 D: opening db index /var/lib/rpm/Triggername (none) mode=0x42 D: opening db index /var/lib/rpm/Dirnames (none) mode=0x42 D: opening db index /var/lib/rpm/Installtid (none) mode=0x42 D: opening db index /var/lib/rpm/Sigmd5 (none) mode=0x42 D: opening db index /var/lib/rpm/Sha1header (none) mode=0x42 D: opening db index /var/lib/rpm/Filetriggername (none) mode=0x42 D: opening db index /var/lib/rpm/Transfiletriggername (none) mode=0x42 D: opening db index /var/lib/rpm/Recommendname (none) mode=0x42 D: opening db index /var/lib/rpm/Suggestname (none) mode=0x42 D: opening db index /var/lib/rpm/Supplementname (none) mode=0x42 D: opening db index /var/lib/rpm/Enhancename (none) mode=0x42 D: computing file dispositions D: 0x0000fd02 4096 51557202 104854262 /var D: read h# 1733 Header SHA1 digest: OK D: Plugin: calling hook psm_pre in selinux plugin D: ========== +++ enroot+caps-3.3.1-1.el7 ppc64le-linux 0x0 D: enroot+caps-3.3.1-1.el7.ppc64le: Header SHA1 digest: OK D: install: enroot+caps-3.3.1-1.el7.ppc64le has 0 files D: Plugin: calling hook psm_pre in selinux plugin Running scriptlet: enroot+caps-3.3.1-1.el7.ppc64le 1/2 fdio: 2 writes, 135 total bytes in 0.000007 secs D: adding "enroot+caps" to Name index. D: adding "Unspecified" to Group index. D: adding 8 entries to Requirename index. D: adding 2 entries to Providename index. D: adding 1 entries to Installtid index. D: adding 1 entries to Sigmd5 index. D: adding "e1359209f621c0442228c0ddbeaecfc136425187" to Sha1header index. D: %post(enroot+caps-3.3.1-1.el7.ppc64le): scriptlet start D: %post(enroot+caps-3.3.1-1.el7.ppc64le): execv(/bin/sh) pid 1137648 D: Plugin: calling hook scriptlet_fork_post in selinux plugin ++ command -v enroot-mksquashovlfs + setcap cap_sys_admin+pe /bin/enroot-mksquashovlfs ++ command -v enroot-aufs2ovlfs + setcap cap_sys_admin,cap_mknod+pe /bin/enroot-aufs2ovlfs D: %post(enroot+caps-3.3.1-1.el7.ppc64le): waitpid(1137648) rc 1137648 status 0 fdio: 6 reads, 3264 total bytes in 0.000007 secs Running scriptlet: enroot+caps-3.3.1-1.el7.ppc64le 2/2 fdio: 2 writes, 135 total bytes in 0.000007 secs D: ========== +++ enroot+caps-3.3.1-1.el7 ppc64le-linux 0x0 D: erase: enroot+caps-3.3.1-1.el7.ppc64le has 0 files D: Plugin: calling hook psm_pre in selinux plugin D: %preun(enroot+caps-3.3.1-1.el7.ppc64le): scriptlet start D: %preun(enroot+caps-3.3.1-1.el7.ppc64le): execv(/bin/sh) pid 1137653 D: Plugin: calling hook scriptlet_fork_post in selinux plugin ++ command -v enroot-mksquashovlfs + setcap cap_sys_admin-pe /bin/enroot-mksquashovlfs ++ command -v enroot-aufs2ovlfs + setcap cap_sys_admin,cap_mknod-pe /bin/enroot-aufs2ovlfs D: %preun(enroot+caps-3.3.1-1.el7.ppc64le): waitpid(1137653) rc 1137653 status 0 Cleanup : enroot+caps-3.3.1-1.el7.ppc64le 2/2 Verifying : enroot+caps-3.3.1-1.el7.ppc64le 1/2 Verifying : enroot+caps-3.3.1-1.el7.ppc64le 2/2 Installed products updated. Reinstalled: enroot+caps-3.3.1-1.el7.ppc64le Complete! [root@ic922-01 ~]# getcap /usr/bin/enroot-aufs2ovlfs /usr/bin/enroot-aufs2ovlfs = [root@ic922-01 ~]# ll /usr/bin/enroot-aufs2ovlfs -rwxr-xr-x 1 root root 67520 Aug 7 02:27 /usr/bin/enroot-aufs2ovlfs [root@ic922-01 ~]# ll /bin/enroot-mksquashovlfs -rwxr-xr-x 1 root root 67520 Aug 7 02:27 /bin/enroot-mksquashovlfs [root@ic922-01 ~]# getcap /usr/bin/enroot-aufs2ovlfs /usr/bin/enroot-aufs2ovlfs = [root@ic922-01 ~]# getcap /bin/enroot-aufs2ovlfs /bin/enroot-aufs2ovlfs = ```

(But seeing that rpm has a setcaps option, maybe there's a part of rpm or spec that is used for this kind of things…)

3XX0 commented 3 years ago

Can you try the following patch, see if it does the trick (including removing the caps on uninstall):

diff --git a/pkg/rpm/SPECS/enroot.spec b/pkg/rpm/SPECS/enroot.spec
index c9d5693..b518f63 100644
--- a/pkg/rpm/SPECS/enroot.spec
+++ b/pkg/rpm/SPECS/enroot.spec
@@ -43,13 +43,9 @@ unprivileged sandboxes.

 This dependency package grants extra capabilities to unprivileged users which
 allows them to import and convert container images directly.
-%post -n %{name}+caps
-setcap cap_sys_admin+pe "$(command -v enroot-mksquashovlfs)"
-setcap cap_sys_admin,cap_mknod+pe "$(command -v enroot-aufs2ovlfs)"
-%preun -n %{name}+caps
-setcap cap_sys_admin-pe "$(command -v enroot-mksquashovlfs)"
-setcap cap_sys_admin,cap_mknod-pe "$(command -v enroot-aufs2ovlfs)"
 %files -n %{name}+caps
+%caps(= cap_sys_admin+ep) %{_bindir}/enroot-mksquashovlfs
+%caps(= cap_sys_admin+ep cap_mknod+ep) %{_bindir}/enroot-aufs2ovlfs

 %build
 %make_build prefix=%{_prefix} exec_prefix=%{_exec_prefix} libdir=%{_libdir} datarootdir=%{_datarootdir}
krono commented 3 years ago

This did not work:

#  dnf install --rpmverbosity=debug dist/ppc64le/enroot*-3.3.1-1.el8.ppc64le.rpm
dio:      16 reads,   135608 total bytes in 0.002284 secs
  Installing       : enroot+caps-3.3.1-1.el8.ppc64le                                                                                                                                                         2/3
D: adding "enroot" to Name index.
D: adding 39 entries to Basenames index.
D: adding "Unspecified" to Group index.
D: adding 23 entries to Requirename index.
D: adding 3 entries to Providename index.
D: adding 14 entries to Dirnames index.
D: adding 1 entries to Installtid index.
D: adding 1 entries to Sigmd5 index.
D: adding "d5bd4fe50e62c2673ccf5d597a0ede50647eb551" to Sha1header index.
D: ========== +++ enroot+caps-3.3.1-1.el8 ppc64le-linux 0x0
D: enroot+caps-3.3.1-1.el8.ppc64le: Header SHA256 digest: OK
D: enroot+caps-3.3.1-1.el8.ppc64le: Header SHA1 digest: OK
D:   install: enroot+caps-3.3.1-1.el8.ppc64le has 2 files
D: Plugin: calling hook psm_pre in selinux plugin
D: skip       100755  1 (   0,   0) 67600 /usr/bin/enroot-aufs2ovlfs;614c2664
D: skip       100755  1 (   0,   0) 67600 /usr/bin/enroot-mksquashovlfs;614c2664
# getcap /bin/enroot-*
# "empty"

however, afterwards doing rpm --setcaps interestingly does work:

# rpm --setcaps enroot+caps
# getcap /bin/enroot-*
/bin/enroot-aufs2ovlfs = cap_sys_admin,cap_mknod+ep
/bin/enroot-mksquashovlfs = cap_sys_admin+ep

weird

3XX0 commented 3 years ago

Not sure what we're missing, it does work on my Centos 7 system with yum Maybe you have nocaps set in dnf.conf?

krono commented 3 years ago

Hey.

Maybe you have nocaps set in dnf.conf? not to my knowledge :/

# cat /etc/dnf/dnf.conf
[main]
gpgcheck=1
installonly_limit=6
clean_requirements_on_remove=True
best=True
skip_if_unavailable=False

I tried yum on RHEL8 and no change.

The debugoutput suggests there is nothing being done, in the first place…

D: ========== +++ enroot+caps-3.3.1-1.el8 ppc64le-linux 0x0
D: enroot+caps-3.3.1-1.el8.ppc64le: Header SHA256 digest: OK
D: enroot+caps-3.3.1-1.el8.ppc64le: Header SHA1 digest: OK
D:   install: enroot+caps-3.3.1-1.el8.ppc64le has 2 files
D: Plugin: calling hook psm_pre in selinux plugin
D: skip       100755  1 (   0,   0) 67600 /usr/bin/enroot-aufs2ovlfs;614c2b8b
D: skip       100755  1 (   0,   0) 67600 /usr/bin/enroot-mksquashovlfs;614c2b8b
krono commented 3 years ago

Ok, this one worked for me:

diff --git a/pkg/rpm/SPECS/enroot.spec b/pkg/rpm/SPECS/enroot.spec
index c9d5693..8c79ed2 100644
--- a/pkg/rpm/SPECS/enroot.spec
+++ b/pkg/rpm/SPECS/enroot.spec
@@ -43,13 +43,21 @@ unprivileged sandboxes.

 This dependency package grants extra capabilities to unprivileged users which
 allows them to import and convert container images directly.
+%files -n %{name}+caps
+%caps(= cap_sys_admin+ep) %{_bindir}/enroot-mksquashovlfs
+%caps(= cap_sys_admin+ep cap_mknod+ep) %{_bindir}/enroot-aufs2ovlfs
 %post -n %{name}+caps
-setcap cap_sys_admin+pe "$(command -v enroot-mksquashovlfs)"
-setcap cap_sys_admin,cap_mknod+pe "$(command -v enroot-aufs2ovlfs)"
+if [ -x /usr/sbin/setcap ]; then
+    /bin/chmod 0755 %{_bindir}/enroot-mksquashovlfs
+    /bin/chmod 0755 %{_bindir}/enroot-aufs2ovlfs
+    /usr/sbin/setcap cap_sys_admin+ep %{_bindir}/enroot-mksquashovlfs
+    /usr/sbin/setcap cap_sys_admin,cap_mknod+pe %{_bindir}/enroot-aufs2ovlfs
+fi
 %preun -n %{name}+caps
-setcap cap_sys_admin-pe "$(command -v enroot-mksquashovlfs)"
-setcap cap_sys_admin,cap_mknod-pe "$(command -v enroot-aufs2ovlfs)"
-%files -n %{name}+caps
+if [ -x /usr/sbin/setcap ]; then
+    /usr/sbin/setcap cap_sys_admin-ep %{_bindir}/enroot-mksquashovlfs
+    /usr/sbin/setcap cap_sys_admin,cap_mknod-pe %{_bindir}/enroot-aufs2ovlfs
+fi

 %build
 %make_build prefix=%{_prefix} exec_prefix=%{_exec_prefix} libdir=%{_libdir} datarootdir=%{_datarootdir}
3XX0 commented 3 years ago

What about just doing

-setcap cap_sys_admin+pe "$(command -v enroot-mksquashovlfs)"
-setcap cap_sys_admin,cap_mknod+pe "$(command -v enroot-aufs2ovlfs)"

+/usr/sbin/setcap cap_sys_admin+ep %{_bindir}/enroot-mksquashovlfs
+/usr/sbin/setcap cap_sys_admin,cap_mknod+pe %{_bindir}/enroot-aufs2ovlfs

Is it enough to make it work? You can also try to add %attr(0755,root,root) in front of each %caps instead, maybe it won't skip it after that

krono commented 3 years ago

I'll try. In fact, I just copy-pasted from https://github.com/schweikert/fping/blob/develop/contrib/fping.spec#L55 to see if it works. I don't think the chmod is necessary in the first place… But I try first

krono commented 3 years ago

Yes, this seems to be enough. Personally, I'd probably also include the %caps for good measure, but the variant with only /usr/sbin/setcap works.

krono commented 3 years ago

And we re open :(

krono commented 3 years ago

I am at loss here

Upgrade log ``` # yum check-update enroot enroot+caps Updating Subscription Management repositories. Last metadata expiration check: 0:02:05 ago on Fri 05 Nov 2021 09:24:49 AM CET. enroot.ppc64le 3.4.0-1.el7 enroot+caps.ppc64le 3.4.0-1.el7 ``` ``` [root ~]# yum update enroot enroot+caps Updating Subscription Management repositories. Last metadata expiration check: 0:02:15 ago on Fri 05 Nov 2021 09:24:49 AM CET. Dependencies resolved. ======================================================== Package Architecture Version Size ======================================================== Upgrading: enroot ppc64le 3.4.0-1.el7 101 k enroot+caps ppc64le 3.4.0-1.el7 3.4 k Transaction Summary ======================================================== Upgrade 2 Packages Total size: 105 k Is this ok [y/N]: y Downloading Packages: Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Upgrading : enroot-3.4.0-1.el7.ppc64le 1/4 Upgrading : enroot+caps-3.4.0-1.el7.ppc64le 2/4 Running scriptlet: enroot+caps-3.4.0-1.el7.ppc64le 2/4 Running scriptlet: enroot+caps-3.3.0-1.el7.ppc64le 3/4 Cleanup : enroot+caps-3.3.0-1.el7.ppc64le 3/4 Cleanup : enroot-3.3.0-1.el7.ppc64le 4/4 Running scriptlet: enroot-3.3.0-1.el7.ppc64le 4/4 Verifying : enroot+caps-3.4.0-1.el7.ppc64le 1/4 Verifying : enroot+caps-3.3.0-1.el7.ppc64le 2/4 Verifying : enroot-3.4.0-1.el7.ppc64le 3/4 Verifying : enroot-3.3.0-1.el7.ppc64le 4/4 Installed products updated. Upgraded: enroot-3.4.0-1.el7.ppc64le enroot+caps-3.4.0-1.el7.ppc64le Complete! ``` ``` [root@ac922-01 ~]# getcap /usr/bin/enroot-aufs2ovlfs /usr/bin/enroot-aufs2ovlfs = ```
krono commented 3 years ago

Workaround that seems to stick: re-install instead of update.

remove/install cycle that works ``` [root ~]# yum remove enroot+caps Updating Subscription Management repositories. Dependencies resolved. ============================================================ Package Architecture Version Repository Size ============================================================ Removing: enroot+caps ppc64le 3.4.0-1.el7 @CONTAINER 0 Transaction Summary ============================================================ Remove 1 Package Freed space: 0 Is this ok [y/N]: y Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Running scriptlet: enroot+caps-3.4.0-1.el7.ppc64le 1/1 Erasing : enroot+caps-3.4.0-1.el7.ppc64le 1/1 Verifying : enroot+caps-3.4.0-1.el7.ppc64le 1/1 Installed products updated. Removed: enroot+caps-3.4.0-1.el7.ppc64le Complete! ``` ``` [root ~]# yum install enroot+caps Updating Subscription Management repositories. Last metadata expiration check: 0:07:35 ago on Fri 05 Nov 2021 09:24:49 AM CET. Dependencies resolved. ============================================================ Package Architecture Version Repository Size ============================================================ Installing: enroot+caps ppc64le 3.4.0-1.el7 CONTAINER 3.4 k Transaction Summary ============================================================ Install 1 Package Total size: 3.4 k Installed size: 0 Is this ok [y/N]: y Downloading Packages: Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : enroot+caps-3.4.0-1.el7.ppc64le 1/1 Running scriptlet: enroot+caps-3.4.0-1.el7.ppc64le 1/1 Verifying : enroot+caps-3.4.0-1.el7.ppc64le 1/1 Installed products updated. Installed: enroot+caps-3.4.0-1.el7.ppc64le Complete! ``` ``` [root ~]# getcap /usr/bin/enroot-aufs2ovlfs /usr/bin/enroot-aufs2ovlfs = cap_sys_admin,cap_mknod+ep ```
krono commented 3 years ago

I presume that running the post-scriplet of uninstall of OLD after the install-scriplet of NEW is the culprit

Running scriptlet: enroot+caps-3.4.0-1.el7.ppc64le   2/4
  Running scriptlet: enroot+caps-3.3.0-1.el7.ppc64le   3/4
  Cleanup          : enroot+caps-3.3.0-1.el7.ppc64le   3/4
  Cleanup          : enroot-3.3.0-1.el7.ppc64le        4/4
  Running scriptlet: enroot-3.3.0-1.el7.ppc64le        4/4
krono commented 3 years ago

I have a hunch that %posttrans instead of %post would help the ordering…

3XX0 commented 3 years ago

Yeah, most likely. I will release new packages once it's fixed