search

coreos / fedora-coreos-tracker

Issue tracker for Fedora CoreOS
https://fedoraproject.org/coreos/
260 stars 61 forks source link

rawhide: util-linux-2.40-0.1.fc40 update is causing kernel-replace test failure #1666

Closed dustymabe closed 4 months ago

dustymabe commented 5 months ago

The util-linux-2.40-0.1.fc40 update seems to have caused a failure in the ext.config.rpm-ostree.kernel-replace test:

systemctl status kola-runext.service:
�� kola-runext.service
     Loaded: loaded (/etc/systemd/system/kola-runext.service; static)
    Drop-In: /usr/lib/systemd/system/service.d
             ������10-timeout-abort.conf
     Active: failed (Result: exit-code) since Mon 2024-02-05 10:12:53 UTC; 310ms ago
   Duration: 44.882s
    Process: 2325 ExecStart=/usr/local/bin/kola-runext-kernel-replace (code=exited, status=1/FAILURE)
   Main PID: 2325 (code=exited, status=1/FAILURE)
        CPU: 41.165s

Feb 05 10:12:09 qemu0 systemd[1]: Started kola-runext.service.
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + . /var/opt/kola/extdata/commonlib.sh
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: ++ IFS=' '
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: ++ read -r -a cmdline
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2326]: ++ mktemp -d
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + cd /tmp/tmp.zUnHF2lkBw
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + image_dir=/var/tmp/fcos
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + image=oci:/var/tmp/fcos
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + image_pull=ostree-unverified-image:oci:/var/tmp/fcos
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + tmp_imagedir=/var/tmp/fcos-tmp
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2327]: ++ arch
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + arch=x86_64
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + kver=6.2.9-300.fc38.x86_64
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + case "${AUTOPKGTEST_REBOOT_MARK:-}" in
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + rpm-ostree status --json
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2334]: ++ jq -r '.deployments[0].checksum'
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + checksum=879f13121f3dfdfc07f72c0f96759f52e56b96c68c82027ae02f6ab94ca2036f
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2336]: ++ jq -r '.deployments[0].version'
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + v0=40.20240205.dev.0
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2338]: ++ jq -r '.deployments[0]["container-image-reference"]'
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + imgref=ostree-remote-image:fedora:docker://quay.io/fedora/fedora-coreos:rawhide
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + rm /var/tmp/fcos -rf
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + encapsulate_args=()
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + [[ ostree-remote-image:fedora:docker://quay.io/fedora/fedora-coreos:rawhide != null ]]
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + encapsulate_args+=("--label" "ostree.bootable=true")
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + systemctl mask --now zincati
Feb 05 10:12:09 qemu0 systemctl[2341]: Created symlink /etc/systemd/system/zincati.service ��� /dev/null.
Feb 05 10:12:09 qemu0 systemctl[2341]: Warning: The unit file, source configuration file or drop-ins of zincati.service changed on disk. Run 'systemctl daemon-reload' to reload units.
Feb 05 10:12:09 qemu0 kola-runext-kernel-replace[2325]: + ostree container encapsulate --label ostree.bootable=true --repo=/ostree/repo 879f13121f3dfdfc07f72c0f96759f52e56b96c68c82027ae02f6ab94ca2036f oci:/var/tmp/fcos
Feb 05 10:12:53 qemu0 kola-runext-kernel-replace[2400]: sha256:58455c2f0b748bf99d3b372df11c7a1813cac6a223c39a0b6431016ab4c29de5
Feb 05 10:12:53 qemu0 kola-runext-kernel-replace[2325]: + rpm-ostree rebase --experimental ostree-unverified-image:oci:/var/tmp/fcos
Feb 05 10:12:53 qemu0 kola-runext-kernel-replace[2413]: Pulling manifest: ostree-unverified-image:oci:/var/tmp/fcos
Feb 05 10:12:53 qemu0 kola-runext-kernel-replace[2413]: error: Creating importer: skopeo proxy unexpectedly exited during request method Initialize: exit status: 127
Feb 05 10:12:53 qemu0 kola-runext-kernel-replace[2413]: setpriv: landlock_create_ruleset failed: No message of desired type
Feb 05 10:12:53 qemu0 systemd[1]: kola-runext.service: Main process exited, code=exited, status=1/FAILURE
Feb 05 10:12:53 qemu0 systemd[1]: kola-runext.service: Failed with result 'exit-code'.
Feb 05 10:12:53 qemu0 systemd[1]: kola-runext.service: Consumed 41.165s CPU time, 4.8M memory peak, 0B memory swap peak.
--- FAIL: ext.config.rpm-ostree.kernel-replace (72.77s)
        cluster.go:162: Error: Unit kola-runext.service exited with code 1
        cluster.go:162: 2024-02-05T10:12:54Z cli: Unit kola-runext.service exited with code 1
        harness.go:1263: kolet failed: : kolet run-test-unit failed: Process exited with status 1

Not sure what the issue is, but will try to pin for now.

cgwalters commented 5 months ago

A brief skim of upstream util-linux issues leads to https://github.com/util-linux/util-linux/issues/2746

dustymabe commented 5 months ago

A brief skim of upstream util-linux issues leads to util-linux/util-linux#2746

Thanks @cgwalters. Last night it was late so I was in full issue+pin mode :)

For my future self the commit that fixes this is https://github.com/util-linux/util-linux/commit/211a9da0b4e9704eea210d7944b723fa78272c79

dustymabe commented 5 months ago

The pin was in https://github.com/coreos/fedora-coreos-config/pull/2842

dustymabe commented 4 months ago

https://github.com/util-linux/util-linux/commit/211a9da0b4e9704eea210d7944b723fa78272c79 was backported to rawhide in https://src.fedoraproject.org/rpms/util-linux/c/1d94d2966ad14311b3ad3c010f1116e81dc7ed42?branch=rawhide so we unpinned in https://github.com/coreos/fedora-coreos-config/pull/2860