search

canonical / lxd

Powerful system container and virtual machine manager
https://canonical.com/lxd
GNU Affero General Public License v3.0
4.32k stars 927 forks source link

Container issues with Ubuntu Oracular systemd version 256-1ubuntu1 #13810

Open mihalicyn opened 1 month ago

mihalicyn commented 1 month ago

Investigate and fix all the issues

TODO list:

See also: https://github.com/canonical/lxd/issues/13807 https://github.com/canonical/lxd/pull/12698 https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/2046486 https://github.com/canonical/lxd/issues/13844 https://github.com/canonical/lxd/issues/13860

tomponline commented 1 month ago

For now the workaround is to do lxc config set <instance> security.nesting=true and restart container.

tomponline commented 1 month ago

Reopens for privilege use case.

mihalicyn commented 1 month ago

Ok, what we have about privileged containers:

What we have in the audit logs:

[15669.318153] audit: type=1400 audit(1721991292.665:4813): apparmor="DENIED" operation="mount" class="mount" info="failed perms check" error=-13 profile="lxd-no_</var/snap/lxd/common/lxd>" name="/run/systemd/mount-rootfs/" pid=145591 comm="(d-logind)" srcname="/" flags="rw, rbind"
[15669.327424] audit: type=1400 audit(1721991292.673:4814): apparmor="DENIED" operation="mount" class="mount" info="failed perms check" error=-13 profile="lxd-no_</var/snap/lxd/common/lxd>" name="/run/systemd/mount-rootfs/" pid=145595 comm="(d-logind)" srcname="/" flags="rw, rbind"
[15669.336943] audit: type=1400 audit(1721991292.685:4815): apparmor="DENIED" operation="mount" class="mount" info="failed perms check" error=-13 profile="lxd-no_</var/snap/lxd/common/lxd>" name="/run/systemd/mount-rootfs/" pid=145599 comm="(d-logind)" srcname="/" flags="rw, rbind"
[15669.346912] audit: type=1400 audit(1721991292.693:4816): apparmor="DENIED" operation="mount" class="mount" info="failed perms check" error=-13 profile="lxd-no_</var/snap/lxd/common/lxd>" name="/run/systemd/mount-rootfs/" pid=145603 comm="(d-logind)" srcname="/" flags="rw, rbind"
[15669.357968] audit: type=1400 audit(1721991292.705:4817): apparmor="DENIED" operation="mount" class="mount" info="failed perms check" error=-13 profile="lxd-no_</var/snap/lxd/common/lxd>" name="/run/systemd/mount-rootfs/" pid=145607 comm="(d-logind)" srcname="/" flags="rw, rbind"
[15671.673932] audit: type=1400 audit(1721991295.021:4818): apparmor="DENIED" operation="mount" class="mount" info="failed perms check" error=-13 profile="lxd-or_</var/snap/lxd/common/lxd>" name="/run/systemd/mount-rootfs/" pid=145649 comm="(d-logind)" srcname="/" flags="rw, rbind"
[15671.674044] audit: type=1400 audit(1721991295.021:4819): apparmor="DENIED" operation="mount" class="mount" info="failed perms check" error=-13 profile="lxd-or_</var/snap/lxd/common/lxd>" name="/dev/" pid=145651 comm="(sd-mkdcreds)" flags="rw, rslave"
[15671.683716] audit: type=1400 audit(1721991295.029:4820): apparmor="DENIED" operation="mount" class="mount" info="failed perms check" error=-13 profile="lxd-or_</var/snap/lxd/common/lxd>" name="/run/systemd/mount-rootfs/" pid=145654 comm="(d-logind)" srcname="/" flags="rw, rbind"
[15671.686683] audit: type=1400 audit(1721991295.033:4821): apparmor="DENIED" operation="mount" class="mount" info="failed perms check" error=-13 profile="lxd-or_</var/snap/lxd/common/lxd>" name="/dev/" pid=145658 comm="(sd-mkdcreds)" flags="rw, rslave"
[15671.692154] audit: type=1400 audit(1721991295.037:4822): apparmor="DENIED" operation="mount" class="mount" info="failed perms check" error=-13 profile="lxd-or_</var/snap/lxd/common/lxd>" name="/run/systemd/mount-rootfs/" pid=145659 comm="(d-logind)" srcname="/" flags="rw, rbind"

This thing wants a recursive bind-mount of / to /run/systemd/mount-rootfs/. While it's perfectly safe for unprivileged containers, we can't allow it for privileged ones. It would be a security breach.

We have this commit in our imagebuilder https://github.com/canonical/lxd-imagebuilder/commit/c928d22574893208fd9ddbdb8c7dcf7e3899c19d and taking into account that images:debian/sid container works with security.privileged=true I would just apply the same quirk to the Ubuntu images.

tomponline commented 1 month ago

Have asked CPC about being able to apply a similar fix as https://github.com/canonical/lxd-imagebuilder/commit/c928d22574893208fd9ddbdb8c7dcf7e3899c19d to ubuntu: images.

https://chat.canonical.com/canonical/pl/dygqosfbp3gy3x7hcs5ssergoy

tomponline commented 1 month ago

@mihalicyn does this fix rely on the apparmor parser version in core24?

mihalicyn commented 1 month ago

@mihalicyn does this fix rely on the apparmor parser version in core24?

yes, but we don't need core24 itself.

Full list of changes we need to backport:

  1. https://github.com/canonical/lxd-pkg-snap/pull/477
  2. https://github.com/canonical/lxd/pull/13681
  3. https://github.com/canonical/lxd/pull/13820 (optional) https://github.com/canonical/lxd/pull/13826
tomponline commented 1 month ago

thanks @mihalicyn

I've added the vendored apparmor parser and your fixes as cherry-picks to latest/stable now.

tomponline commented 2 weeks ago

We are rolling out fixes to 5.21/stable for this now (snap refresh lxd --channel=5.21/stable --cohort="+" will get it by bypassing progressive rollout).