search

lxc / incus

Powerful system container and virtual machine manager
https://linuxcontainers.org/incus
Apache License 2.0
2.5k stars 203 forks source link

Snaps not running inside containers #445

Closed loxK closed 8 months ago

loxK commented 8 months ago

Required information

Issue description

I migrated from LXD. After that snaps are not running any more inside all my containers:

root@front:~# snap list
Name                 Version   Rev    Tracking       Publisher     Notes
certbot              2.8.0     3566   latest/stable  certbot-eff✓  classic
certbot-dns-rfc2136  2.8.0     3150   latest/beta    certbot-eff✓  -
core18               20231027  2812   latest/stable  canonical✓    base
core20               20231123  2105   latest/stable  canonical✓    base
snapd                2.61.1    20671  latest/stable  canonical✓    snapd
root@front:~# certbot renew
missing profile snap.certbot.certbot.
Please make sure that the snapd.apparmor service is enabled and started

Snapd's running:

# systemctl status snapd.service 
● snapd.service - Snap Daemon
     Loaded: loaded (/lib/systemd/system/snapd.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2024-01-28 11:58:34 +11; 6s ago
TriggeredBy: ● snapd.socket
   Main PID: 287 (snapd)
      Tasks: 11 (limit: 38144)
     Memory: 54.5M
     CGroup: /system.slice/snapd.service
             └─287 /usr/lib/snapd/snapd

janv. 28 11:58:29 front.domain.io systemd[1]: Starting Snap Daemon...
janv. 28 11:58:32 front.domain.io snapd[287]: overlord.go:271: Acquiring state lock file
janv. 28 11:58:32 front.domain.io snapd[287]: overlord.go:276: Acquired state lock file
janv. 28 11:58:33 front.domain.io snapd[287]: daemon.go:247: started snapd/2.61.1 (series 16; classic) ubuntu/20.04 (amd64) linux/5.15.0-92-generic.
janv. 28 11:58:33 front.domain.io snapd[287]: daemon.go:340: adjusting startup timeout by 55s (pessimistic estimate of 30s plus 5s per snap)
janv. 28 11:58:33 front.domain.io snapd[287]: backends.go:58: AppArmor status: apparmor is enabled and all features are available (using snapd provided apparmor_parser)
janv. 28 11:58:34 front.domain.io systemd[1]: Started Snap Daemon.
# systemctl status snapd.apparmor.service 
● snapd.apparmor.service - Load AppArmor profiles managed internally by snapd
     Loaded: loaded (/lib/systemd/system/snapd.apparmor.service; enabled; vendor preset: enabled)
     Active: active (exited) since Sun 2024-01-28 11:58:26 +11; 5min ago
   Main PID: 232 (code=exited, status=0/SUCCESS)
      Tasks: 0 (limit: 38144)
     Memory: 0B
     CGroup: /system.slice/snapd.apparmor.service

janv. 28 11:58:26 front.domain.io systemd[1]: Starting Load AppArmor profiles managed internally by snapd...
janv. 28 11:58:26 front.domain.io snapd-apparmor[232]: main.go:166: Inside container environment without internal policy
janv. 28 11:58:26 front.domain.io systemd[1]: Finished Load AppArmor profiles managed internally by snapd.

journalctl before migration

-- Reboot --
janv. 28 05:09:45 front.domain.io systemd[1]: Starting Load AppArmor profiles managed internally by snapd...
janv. 28 05:09:46 front.domain.io snapd-apparmor[222]: main.go:124: Loading profiles [/var/lib/snapd/apparmor/profiles/snap-confine.snapd.20290 /var/lib/snapd/apparmor/profi>
janv. 28 05:09:47 front.domain.io systemd[1]: Finished Load AppArmor profiles managed internally by snapd.

After migration

-- Reboot --
janv. 28 11:07:16 front.domain.io systemd[1]: Starting Load AppArmor profiles managed internally by snapd...
janv. 28 11:07:17 front.domain.io snapd-apparmor[211]: main.go:166: Inside container environment without internal policy
janv. 28 11:07:17 front.domain.io systemd[1]: Finished Load AppArmor profiles managed internally by snapd.

Steps to reproduce

  1. Migrate from LXD to Incus
  2. incus shell mycontainer
  3. try a command from a snap installed app

Information to attach

loxK commented 8 months ago

It looks like it has been fixed upstream : https://github.com/snapcore/snapd/pull/13350

Do I have to wait for a new snapd release ? Shouldn't it need to be specified in migration guide ?

stgraber commented 8 months ago

There is a workaround that you can use until apparmor and snapd are both updated with the fix.

https://discuss.linuxcontainers.org/t/weird-snap-behavior-it-is-there-but-won-t-work-i-tried-a-couple-of-different-containers-will-try-on-my-home-machines-later-did-the-snaps-break-all-of-a-sudden-or-is-it-my-imagination-they-worked-before-in-a-container/18682/2?u=stgraber

stgraber commented 8 months ago

Closing as there isn't really anything we can do about it in Incus until the fixes start making their way into Ubuntu images.

stgraber commented 8 months ago

The post linked above shows snaps working on Incus, but for that to be the case given the current state of apparmor and snapd requires manually updating the apparmor startup script to treat Incus as it does LXD.

Because that same trick can't be applied to the equivalent snapd binary, the trick on the snapd side is to pretend that the system is Microsoft' Windows Subsystem for Linux, which then causes snapd to move on and work inside the container.

I wrote and sent an upstream fix to AppArmor a few months ago which got quickly included upstream but hasn't yet made it into Ubuntu.

For snapd, I reported the issue but because of the Canonical CLA being applied to the snapd package, I couldn't send a fix myself. Thankfully, someone else did, so this will also eventually make it into Ubuntu.

loxK commented 7 months ago

That is annoying as AppArmor is many versions behind the latest release, even in Jammy. 3.0.4 was released a year ago.

» dpkg -l | grep apparmor
ii  apparmor                                   3.0.4-2ubuntu2.3                             amd64        user-space parser utility for AppArmor
ii  libapparmor1:amd64                         3.0.4-2ubuntu2.3                             amd64        changehat AppArmor library

Focal is still using 2.13 version and the fix hasn't been back ported to that version (only 3.0 and 3.1)

I think that should be mentioned in the migration guide as it may be an important thing sysadmin should take into account. Personally, I use Snap for easy Certbot and Docker install.