package-update-upgrade-install does not update when run with --frequency=always

[ubuntu-server-builder]

This bug was originally filed in Launchpad as LP: #1785225

Launchpad details

affected_projects = []
assignee = None
assignee_name = None
date_closed = None
date_created = 2018-08-03T11:12:56.923746+00:00
date_fix_committed = None
date_fix_released = None
id = 1785225
importance = medium
is_complete = False
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1785225
milestone = None
owner = siriobalmelli
owner_name = Sirio Balmelli
private = False
status = triaged
submitter = siriobalmelli
submitter_name = Sirio Balmelli
tags = []
duplicates = []

Launchpad user Sirio Balmelli(siriobalmelli) wrote on 2018-08-03T11:12:56.923746+00:00

1. The following stanzas in /etc/cloud/cloud.cfg:

cloud_final_modules:
  # always update+upgrade on boot
  - [package-update-upgrade-install, always]

packages:  # will run apt-update
  - git
package_update: true
package_upgrade: true
package_reboot_if_required: true

2. Result in the following log entry at boot-time:

helpers.py[DEBUG]: update-sources already ran (freq=once-per-instance)

3. Running the module manually with:

sudo cloud-init single --name package-update-upgrade-install --frequency always

... also fails to call apt-get update

4. Google gives no results.

   • No similar bugs visible here.
   • No additional configuration keys shown in https://cloudinit.readthedocs.io/en/latest/topics/modules.html#package-update-upgrade-install.
   • No relevant keys given in https://cloudinit.readthedocs.io/en/latest/topics/modules.html#apt-configure.
   • Nowhere is "update-sources" mentioned.

5. Attempted guess at "update-sources" fails:

sudo cloud-init single --name update-sources --frequency always

6. Expected behavior is that apt-get update is executed before apt-get install every time the machine is booted. On my system the install looks like:

['eatmydata', 'apt-get', '--option=Dpkg::Options::=--force-confold', '--option=Dpkg::options::=--force-unsafe-io', '--assume-yes', '--quiet', 'install', 'git']

7. Alternate behavior would be a way to explicitly request that an update be performed. This was attempted/guessed-at with:

cloud_config_modules:
  - [apt-configure, always]

But did not obtain an update. There seem to be no other applicable modules listed in https://cloudinit.readthedocs.io/en/latest/topics/modules.html#

8. Environment:

   • Ubuntu 16.04.5 LTS
   • /usr/bin/cloud-init 18.2

9. /etc/cloud/cloud.cfg

   cloud_init_modules:   - seed_random   - bootcmd   - set_hostname   - update_hostname   - update_etc_hosts   - ca-certs   - rsyslog   - users-groups   - ssh

cloud_config_modules:   - emit_upstart   - ssh-import-id   - locale   - set-passwords   - grub-dpkg   - timezone   - [apt-configure, always]   - [runcmd, always] # doesn't actually EXECUTE the modules - that's left for scripts-user :P

cloud_final_modules:   - landscape   - lxd   - ssh-authkey-fingerprints   - final-message   - power-state-change   - [package-update-upgrade-install, always]   - [scripts-user, always]

system_info:   distro: ubuntu   paths:     cloud_dir: /var/lib/cloud/     templates_dir: /etc/cloud/templates/     upstart_dir: /etc/init/   ssh_svcname: ssh

disable_root: true

datasource_list: [NoCloud, None]

preserve_hostname: false hostname: {hostname} manage_etc_hosts: true

apt:   preserve_sources_list: false   primary:     - arches:         - amd64         - i386         - default       uri: "http://archive.ubuntu.com/ubuntu/"       search:         - "http://ch.archive.ubuntu.com/ubuntu"         - "http://us.archive.ubuntu.com/ubuntu"       search_dns: true   security:     - arches:         - amd64         - i386         - default       uri: "http://security.ubuntu.com/ubuntu/"       search_dns: true   # (custom PPAs elided)   sources_list: |     deb http://archive.ubuntu.com/ubuntu/ xenial main restricted     deb-src http://archive.ubuntu.com/ubuntu/ xenial main restricted     deb http://archive.ubuntu.com/ubuntu/ xenial-updates main restricted     deb-src http://archive.ubuntu.com/ubuntu/ xenial-updates main restricted     deb http://archive.ubuntu.com/ubuntu/ xenial universe     deb-src http://archive.ubuntu.com/ubuntu/ xenial universe     deb http://archive.ubuntu.com/ubuntu/ xenial-updates universe     deb-src http://archive.ubuntu.com/ubuntu/ xenial-updates universe     deb http://archive.ubuntu.com/ubuntu/ xenial multiverse     deb-src http://archive.ubuntu.com/ubuntu/ xenial multiverse     deb http://archive.ubuntu.com/ubuntu/ xenial-updates multiverse     deb-src http://archive.ubuntu.com/ubuntu/ xenial-updates multiverse     deb http://archive.ubuntu.com/ubuntu/ xenial-backports main restricted universe multiverse     deb-src http://archive.ubuntu.com/ubuntu/ xenial-backports main restricted universe multiverse     deb http://security.ubuntu.com/ubuntu/ xenial-security main restricted     deb-src http://security.ubuntu.com/ubuntu/ xenial-security main restricted     deb http://security.ubuntu.com/ubuntu/ xenial-security universe     deb-src http://security.ubuntu.com/ubuntu/ xenial-security universe     deb http://security.ubuntu.com/ubuntu/ xenial-security multiverse     deb-src http://security.ubuntu.com/ubuntu/ xenial-security multiverse runcmd:   - [echo, startup run $(date)]

packages: # SHOULD run apt-update; does not   - git package_update: true package_upgrade: true package_reboot_if_required: true

[ubuntu-server-builder]

Launchpad user Ryan Harper(raharper) wrote on 2019-07-19T17:46:56.002900+00:00

Hi,

Thanks for filing the bug.

It appears that the distro.update_package_sources() runs with hardcoded frequency=PER_INSTANCE.

def update_package_sources(self):                                          
    self._runner.run("update-sources", self.package_command,               
                     ["update"], freq=PER_INSTANCE)

Cloud-init could supply the package-update-upgrade frequency when calling this distro method if configured.

[adrien-n]

Hi,

I was trying to add a PPA and install packages from it but it seems such scenarios are made impossible by this issue because it's not possible to "apt update" afterwards.

My config is at https://gitlab.com/crypto-config/crypto-configuration/-/blob/main/lxd-profile if you wish to look at it.

I'm not familiar with the cloud-init code and I don't know if freq can handle that but I would expect there should be a variable that is set after apt update, and unset whenever there is an operation that touches sources.

[blackboxsw]

Thanks @adrien-n for the update and reproducer steps here. I think something like the following is what we want in this case. we want to force apt update to run if previously something like custom apt sources were provided (which triggered that first apt update call and sets theupdate-sources semaphore). Here's a very basic diff that only fnuctionally handles debian.ubuntu, but we'd need to reflect update_package_sources call signatures to all distributions which subclass this method to ensure we don't traceback when freq param is provided.

diff --git a/cloudinit/config/cc_package_update_upgrade_install.py b/cloudinit/config/cc_package_update_upgrade_install.py
index a26e001d3..f62819e9d 100644
--- a/cloudinit/config/cc_package_update_upgrade_install.py
+++ b/cloudinit/config/cc_package_update_upgrade_install.py
@@ -17,7 +17,7 @@ from cloudinit.config import Config
 from cloudinit.config.schema import MetaSchema, get_meta_doc
 from cloudinit.distros import ALL_DISTROS
 from cloudinit.log import flush_loggers
-from cloudinit.settings import PER_INSTANCE
+from cloudinit.settings import PER_ALWAYS, PER_INSTANCE

 REBOOT_FILE = "/var/run/reboot-required"
 REBOOT_CMD = ["/sbin/reboot"]
@@ -104,7 +104,13 @@ def handle(name: str, cfg: Config, cloud: Cloud, args: list) -> None:
     errors = []
     if update or upgrade:
         try:
-            cloud.distro.update_package_sources()
+            # We may want to only provide PER_ALWAYS if we also know
+            # update_packages was already performed due to user-data or system
+            # configuration in /etc/cloud/cloud.cfg.*/
+            kwargs = {}
+            if cloud.distro.runner.has_run("update-pacakges", PER_INSTANCE):
+                kwargs["freq"] = PER_ALWAYS
+            cloud.distro.update_package_sources(**kwargs)
         except Exception as e:
             util.logexc(LOG, "Package update failed")
             errors.append(e)
diff --git a/cloudinit/distros/__init__.py b/cloudinit/distros/__init__.py
index 87390f634..85988db75 100644
--- a/cloudinit/distros/__init__.py
+++ b/cloudinit/distros/__init__.py
@@ -37,6 +37,7 @@ from cloudinit import (
     importer,
     net,
     persistence,
+    settings,
     ssh_util,
     subp,
     temp_utils,
@@ -322,10 +323,12 @@ class Distro(persistence.CloudInitPickleMixin, metaclass=abc.ABCMeta):
         # managers.
         raise NotImplementedError()

-    def update_package_sources(self):
+    def update_package_sources(
+        self, freq: str = settings.PER_INSTANCE
+    ):
         for manager in self.package_managers:
             try:
-                manager.update_package_sources()
+                manager.update_package_sources(freq=freq)
             except Exception as e:
                 LOG.error(
                     "Failed to update package using %s: %s", manager.name, e
diff --git a/cloudinit/distros/package_management/apt.py b/cloudinit/distros/package_management/apt.py
index 3c4164cd2..378fb0888 100644
--- a/cloudinit/distros/package_management/apt.py
+++ b/cloudinit/distros/package_management/apt.py
@@ -105,12 +105,12 @@ class Apt(PackageManager):
             apt_get_upgrade_subcommand=cfg.get("apt_get_upgrade_subcommand"),
         )

-    def update_package_sources(self):
+    def update_package_sources(self, freq: str = PER_INSTANCE):
         self.runner.run(
             "update-sources",
             self.run_package_command,
             ["update"],
-            freq=PER_INSTANCE,
+            freq=freq,
         )

     @functools.lru_cache(maxsize=1)
diff --git a/cloudinit/distros/package_management/package_manager.py b/cloudinit/distros/package_management/package_manager.py
index 864555f6a..381e9673b 100644
--- a/cloudinit/distros/package_management/package_manager.py
+++ b/cloudinit/distros/package_management/package_manager.py
@@ -18,7 +18,7 @@ class PackageManager(ABC):
         return cls(runner)

     @abstractmethod
-    def update_package_sources(self):
+    def update_package_sources(self, freq: str):
         ...

     @abstractmethod
diff --git a/cloudinit/distros/package_management/snap.py b/cloudinit/distros/package_management/snap.py
index 92eb1af8f..582cf29ab 100644
--- a/cloudinit/distros/package_management/snap.py
+++ b/cloudinit/distros/package_management/snap.py
@@ -1,6 +1,6 @@
 # This file is part of cloud-init. See LICENSE file for license information.
 import logging
-from typing import Iterable, List
+from typing import Iterable, List, Optional

 from cloudinit import subp, util
 from cloudinit.distros.package_management.package_manager import (
@@ -14,7 +14,7 @@ LOG = logging.getLogger(__name__)
 class Snap(PackageManager):
     name = "snap"

-    def update_package_sources(self):
+    def update_package_sources(self, freq: Optional[str]):
         pass

     def install_packages(self, pkglist: Iterable) -> UninstalledPackages:

Add hacktoberfest label on this issue as I think it is reasonable for anyone to work this issue.

[Foxboron]

The 23.04 cloud images have now been pulled and we can't deploy our stuff with 23.10 until this has been fixed.

[Foxboron]

The Azure Ubuntu images suffer from having a predefined list of mirrors in /etc/cloud/cloud.cfg.d/90-azure.cfg which means that you can't provision these images with your own images and install packages properly. As this is only present in 23.10 currently I suspect this issue is going to be important with the release of the next Ubuntu LTS 24.04 as people will need to actively work around this for all their Azure Ubuntu installs.

To work around this issue you need to replace the package directive with a runcdm that contains apt-get -y update and then use apt-get install -y for all the packages you need.

Alternatively you need to run apt-get -y update in a runcmd before the packages directive.