search

canonical / snapcraft

Package, distribute, and update any app for Linux and IoT.
https://snapcraft.io
GNU General Public License v3.0
1.19k stars 447 forks source link

snapcraft --destructive-mode fails without sudo #5113

Open basak opened 1 month ago

basak commented 1 month ago

Bug Description

If I run snapcraft --destructive-mode, it fails.

To Reproduce

On a fresh Noble VM:

sudo snap install --classic snapcraft
# installs 8.4.3 (12823)
mkdir -p demo/snap
cat > demo/snap/snapcraft.yaml <<EOT
name: demo
version: 1
summary: demo
description: demo
confinement: classic
grade: devel
base: core24

apps:
  demo:
    command: usr/bin/bash

parts:
  demo:
    build-attributes: [enable-patchelf]
    plugin: nil
    stage-packages: [bash]

package-repositories:
  - type: apt
    url: https://ppa.launchpadcontent.net/racb/experimental3/ubuntu
    suites: [noble]
    components: [main]
    key-id: D4D3DAC83AF800B5BB2C5407691DFAAA8A4EFD6A
EOT
cd demo
snapcraft --destructive-mode

Expected results: success Actual results: Unknown error: Failed to install GPG key: Full execution log: '/home/ubuntu/.local/state/snapcraft/log/snapcraft-20241013-093507.964064.log'

Environment

Ubuntu 24.04 cloud image, snapcraft in destructive mode. Snapcraft snap 8.4.3 (12823).

snapcraft.yaml

name: demo
version: 1
summary: demo
description: demo
confinement: classic
grade: devel
base: core24

apps:
  demo:
    command: usr/bin/bash

parts:
  demo:
    build-attributes: [enable-patchelf]
    plugin: nil
    stage-packages: [bash]

package-repositories:
  - type: apt
    url: https://ppa.launchpadcontent.net/racb/experimental3/ubuntu
    suites: [noble]
    components: [main]
    key-id: D4D3DAC83AF800B5BB2C5407691DFAAA8A4EFD6A

Relevant log output

2024-10-13 09:35:07.964 Starting snapcraft, version 8.4.3
2024-10-13 09:35:07.964 Log verbosity level set to BRIEF
2024-10-13 09:35:07.964 Preparing application...
2024-10-13 09:35:07.965 Configuring application...
2024-10-13 09:35:07.966 Setting up ConfigService
2024-10-13 09:35:07.992 Build plan: platform=None, build_for=None
2024-10-13 09:35:07.992 Not running managed mode because `--destructive-mode` was passed
2024-10-13 09:35:07.992 Loading project because a directory was not provided.
2024-10-13 09:35:07.993 Loading project file '/home/ubuntu/demo/snap/snapcraft.yaml'
2024-10-13 09:35:08.049 Processing grammar (on amd64 for amd64)
2024-10-13 09:35:08.049 Processing grammar for build-attributes: ['enable-patchelf']
2024-10-13 09:35:08.050 Processing grammar for plugin: nil
2024-10-13 09:35:08.050 Processing grammar for stage-packages: ['bash']
2024-10-13 09:35:08.056 Running snapcraft pack on host
2024-10-13 09:35:12.446 Initialising lifecycle manager in /home/ubuntu/demo
2024-10-13 09:35:12.463 Project vars: {'version': '1', 'grade': 'devel'}
2024-10-13 09:35:12.463 Adopting part: None
2024-10-13 09:35:12.467 Using parallel build count of 1 from CPU count
2024-10-13 09:35:12.473 is_snap: True, SNAP_NAME set to snapcraft
2024-10-13 09:35:12.563 Keyring file not found: /etc/apt/keyrings/craft-8A4EFD6A.gpg
2024-10-13 09:35:12.564 Executing command: ['gpg', '--batch', '--no-default-keyring', '--with-colons', '--keyring', 'gnupg-ring:/etc/apt/keyrings/craft-8A4EFD6A.gpg', '--homedir', '/tmp/tmpp4tvfp0_', '--keyserver', 'keyserver.ubuntu.com', '--recv-keys', 'D4D3DAC83AF800B5BB2C5407691DFAAA8A4EFD6A']
2024-10-13 09:35:12.756 Unknown error: Failed to install GPG key:
2024-10-13 09:35:12.765 Traceback (most recent call last):
2024-10-13 09:35:12.765   File "/snap/snapcraft/12823/lib/python3.10/site-packages/craft_archives/repo/apt_key_manager.py", line 273, in install_key_from_keyserver
2024-10-13 09:35:12.765     _call_gpg(
2024-10-13 09:35:12.765   File "/snap/snapcraft/12823/lib/python3.10/site-packages/craft_archives/repo/apt_key_manager.py", line 56, in _call_gpg
2024-10-13 09:35:12.765     process = subprocess.run(
2024-10-13 09:35:12.765   File "/snap/snapcraft/12823/usr/lib/python3.10/subprocess.py", line 526, in run
2024-10-13 09:35:12.765     raise CalledProcessError(retcode, process.args,
2024-10-13 09:35:12.765 subprocess.CalledProcessError: Command '['gpg', '--batch', '--no-default-keyring', '--with-colons', '--keyring', 'gnupg-ring:/etc/apt/keyrings/craft-8A4EFD6A.gpg', '--homedir', '/tmp/tmpp4tvfp0_', '--keyserver', 'keyserver.ubuntu.com', '--recv-keys', 'D4D3DAC83AF800B5BB2C5407691DFAAA8A4EFD6A']' returned non-zero exit status 2.
2024-10-13 09:35:12.765 
2024-10-13 09:35:12.765 During handling of the above exception, another exception occurred:
2024-10-13 09:35:12.765 Traceback (most recent call last):
2024-10-13 09:35:12.765   File "/snap/snapcraft/12823/lib/python3.10/site-packages/craft_application/services/lifecycle.py", line 249, in run
2024-10-13 09:35:12.765     repositories.install_package_repositories(
2024-10-13 09:35:12.765   File "/snap/snapcraft/12823/lib/python3.10/site-packages/craft_application/util/repositories.py", line 51, in install_package_repositories
2024-10-13 09:35:12.765     refresh_required = repo.install(package_repositories, key_assets=key_assets)
2024-10-13 09:35:12.765   File "/snap/snapcraft/12823/lib/python3.10/site-packages/craft_archives/repo/installer.py", line 44, in install
2024-10-13 09:35:12.765     return _install_repos(
2024-10-13 09:35:12.765   File "/snap/snapcraft/12823/lib/python3.10/site-packages/craft_archives/repo/installer.py", line 91, in _install_repos
2024-10-13 09:35:12.765     refresh_required |= key_manager.install_package_repository_key(
2024-10-13 09:35:12.765   File "/snap/snapcraft/12823/lib/python3.10/site-packages/craft_archives/repo/apt_key_manager.py", line 334, in install_package_repository_key
2024-10-13 09:35:12.765     self.install_key_from_keyserver(key_id=key_id, key_server=key_server)
2024-10-13 09:35:12.765   File "/snap/snapcraft/12823/lib/python3.10/site-packages/craft_archives/repo/apt_key_manager.py", line 284, in install_key_from_keyserver
2024-10-13 09:35:12.765     raise errors.AptGPGKeyInstallError(
2024-10-13 09:35:12.765 craft_archives.repo.errors.AptGPGKeyInstallError: Failed to install GPG key:
2024-10-13 09:35:12.765 Full execution log: '/home/ubuntu/.local/state/snapcraft/log/snapcraft-20241013-093507.964064.log'

Additional context

On the other hand, sudo snapcraft --destructive-mode works. But the requirement for this isn't documented anywhere (eg. no mention at https://snapcraft.io/docs/build-options where --destructive-mode is documented). I assume this requirement is not intended? Otherwise snapcraft --destructive-mode should simply fail to run immediately.

This seems to be a regression between base: core20 and base: core24.

basak commented 1 month ago

Further context: I found this when trying to move the git-ubuntu snap to core24. That's complex enough so I came up with a minimal reproducer. The minimal reproducer didn't require me to actually stage any packages from the apt repository, since it fails before that stage, so I just used an arbitrary repository I have around.

I think this isn't blocking git-ubuntu since it's built in CI in destructive mode and I can probably use sudo there without problems (I haven't tried yet), but I'll leave a link to this issue in my workaround.

basak commented 1 month ago

Further, if I drop the package-repositories key, then the same production steps fail with: Error installing snap 'core24' from channel 'latest/stable'. unless I use sudo. Speculation: this seems to be another symptom of the same bug.

mr-cal commented 1 month ago

Hi @basak,

I think the problem is with the documentation. It doesn't explain the expectations for the environment where Snapcraft can run in destructive mode.

Building core20 and older snaps in destructive mode would detect if sudo needed to be used for most operations. Starting in core22, this was dropped. Now, the burden of setting up the environment so there are permissions to do things like install snaps, packages, apt repositories falls on the user. Most environments where destructive mode is used run as root.

I updated the documentation https://snapcraft.io/docs/build-options to be more clear.

basak commented 1 month ago

Thanks @mr-cal that's very helpful!

I have a couple of suggestions:

  1. Perhaps snapcraft --destructive-mode should just fail immediately if it's not root then? That would really help with development and debugging, rather than seeing an error message that doesn't make the failure reason obvious. I appreciate that perhaps there might be a few cases where it would work anyway, but it doesn't seem worth it to me to support that for the downside of hiding the root cause as a consequence, but I leave that to your judgement.
  2. "While destructive mode can be used in CI to save time, using a LXD container is still recommended" would be huge pain when CI is already setting up a VM for the purpose! Otherwise lxd init needs automating, lxd needs to download an additional image every time, etc. I can do these things, but getting something to work automatically in a remote environment for the first time is always time consuming. Would you consider changing your recommendation stating that it's OK to do in a fresh VM please? Or do you plan to drop --destructive-mode in the future entirely?
mr-cal commented 1 month ago
  1. Checking for root access is a good idea. We have some precedence for this already. Like you said, some snaps can be built without root permissions, so this could just be a warning rather than a failure for snapcraft 8.
  2. This is a good point and I forgot to mention in the docs; I just updated them. Destructive mode isn't going away in the short term, but we are favoring containerized builds, which should be a good solution for CI. We have https://github.com/canonical/snapcraft-rocks, although it still has some rough edges.