clearlinux / distribution

Placeholder repository to allow filing of general bugs/issues/etc against the Clear Linux OS for Intel Architecture linux distribution
520 stars 29 forks source link

Clear Linux based docker images failing after updating #400

Closed bryteise closed 5 years ago

bryteise commented 5 years ago

Describe the bug After creating a docker image based on Clear Linux that updates itself, the image can not be run and fails with a unable to exec command.

To Reproduce Steps to reproduce the behavior:

  1. (outside container) docker run -it clearlinux
  2. (in container) swupd update
  3. (outside of container) docker commit $container-id
  4. (outside of container) docker run -it $new-container-id /bin/bash

Expected behavior A clear and concise description of what you expected to happen. New container runs as expected.

Screenshots If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

Additional context Add any other context about the problem here.

lsandov1 commented 5 years ago

One way to easily reproduce the issue: https://github.com/lsandoval/docker-swupd-update This is the error: "standard_init_linux.go:190: exec user process caused "no such file or directory""

mrkz commented 5 years ago

FWIW: This is the debug output of dockerd --debug when running new commited CLR image (docker run -it updated_clr):

Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.468886577-06:00" level=debug msg="Calling GET /_ping"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.470129964-06:00" level=debug msg="Calling POST /v1.38/containers/create"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.470298841-06:00" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":true,\"AttachStdout\":true,\"Cmd\":null,\"Domainname\":\"\",\"Entrypoint\":null,\"Env\":[],\"HostConfig\":{\"AutoRemove\":false,\"Binds\":null,\"BlkioDeviceReadBps\":null,\"BlkioDeviceReadIOps\":null,\"BlkioDeviceWriteBps\":null,\"BlkioDeviceWriteIOps\":null,\"BlkioWeight\":0,\"BlkioWeightDevice\":[],\"CapAdd\":null,\"CapDrop\":null,\"Cgroup\":\"\",\"CgroupParent\":\"\",\"ConsoleSize\":[0,0],\"ContainerIDFile\":\"\",\"CpuCount\":0,\"CpuPercent\":0,\"CpuPeriod\":0,\"CpuQuota\":0,\"CpuRealtimePeriod\":0,\"CpuRealtimeRuntime\":0,\"CpuShares\":0,\"CpusetCpus\":\"\",\"CpusetMems\":\"\",\"DeviceCgroupRules\":null,\"Devices\":[],\"DiskQuota\":0,\"Dns\":[],\"DnsOptions\":[],\"DnsSearch\":[],\"ExtraHosts\":null,\"GroupAdd\":null,\"IOMaximumBandwidth\":0,\"IOMaximumIOps\":0,\"IpcMode\":\"\",\"Isolation\":\"\",\"KernelMemory\":0,\"Links\":null,\"LogConfig\":{\"Config\":{},\"Type\":\"\"},\"MaskedPaths\":null,\"Memory\":0,\"MemoryReservation\":0,\"MemorySwap\":0,\"MemorySwappiness\":-1,\"NanoCpus\":0,\"NetworkMode\":\"default\",\"OomKillDisable\":false,\"OomScoreAdj\":0,\"PidMode\":\"\",\"PidsLimit\":0,\"PortBindings\":{},\"Privileged\":false,\"PublishAllPorts\":false,\"ReadonlyPaths\":null,\"ReadonlyRootfs\":false,\"RestartPolicy\":{\"MaximumRetryCount\":0,\"Name\":\"no\"},\"SecurityOpt\":null,\"ShmSize\":0,\"UTSMode\":\"\",\"Ulimits\":null,\"UsernsMode\":\"\",\"VolumeDriver\":\"\",\"VolumesFrom\":null},\"Hostname\":\"\",\"Image\":\"updated_clr\",\"Labels\":{},\"NetworkingConfig\":{\"EndpointsConfig\":{}},\"OnBuild\":null,\"OpenStdin\":true,\"StdinOnce\":true,\"Tty\":true,\"User\":\"\",\"Volumes\":{},\"WorkingDir\":\"\"}"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.499448693-06:00" level=debug msg="container mounted via layerStore: &{/var/lib/docker/overlay2/825f56dc0f463eee0313d20940511d627f9193b70fbe42fe94d5970efe3a1566/merged 0x55d3e0059980 0x55d3e0059980}"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.513379916-06:00" level=debug msg="Calling POST /v1.38/containers/9dd2b629320270c4d84df5d85e0bb0930deb3e6e8b63d933ceec3120bdbc682d/attach?stderr=1&stdin=1&stdout=1&stream=1"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.513483337-06:00" level=debug msg="attach: stderr: begin"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.513503312-06:00" level=debug msg="attach: stdout: begin"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.513503220-06:00" level=debug msg="attach: stdin: begin"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.513710703-06:00" level=debug msg="Calling POST /v1.38/containers/9dd2b629320270c4d84df5d85e0bb0930deb3e6e8b63d933ceec3120bdbc682d/wait?condition=next-exit"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.514075465-06:00" level=debug msg="Calling POST /v1.38/containers/9dd2b629320270c4d84df5d85e0bb0930deb3e6e8b63d933ceec3120bdbc682d/start"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.514865898-06:00" level=debug msg="container mounted via layerStore: &{/var/lib/docker/overlay2/825f56dc0f463eee0313d20940511d627f9193b70fbe42fe94d5970efe3a1566/merged 0x55d3e0059980 0x55d3e0059980}"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.515119348-06:00" level=debug msg="Assigning addresses for endpoint tender_goldstine's interface on network bridge"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.515136160-06:00" level=debug msg="RequestAddress(LocalDefault/172.17.0.0/16, <nil>, map[])"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.515157695-06:00" level=debug msg="Request address PoolID:172.17.0.0/16 App:
ipam/default/data, ID: LocalDefault/172.17.0.0/16, DBIndex: 0x0, Bits: 65536, Unselected: 65533, Sequence: (0xc0000000, 1)->(0x0, 2046)->(0x1, 1)->end Curr:3 Serial:false PrefAddress:<nil> "
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.521479368-06:00" level=debug msg="Assigning addresses for endpoint tender_goldstine's interface on network bridge"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.528840632-06:00" level=debug msg="Programming external connectivity on endpoint tender_goldstine (82cd17f08207141d15fd9570403c400e540340894d1dad5d5f2d871076c3d6d1)"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.530784767-06:00" level=debug msg="EnableService 9dd2b629320270c4d84df5d85e0bb0930deb3e6e8b63d933ceec3120bdbc682d START"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.530797004-06:00" level=debug msg="EnableService 9dd2b629320270c4d84df5d85e0bb0930deb3e6e8b63d933ceec3120bdbc682d DONE"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.533701848-06:00" level=debug msg="bundle dir created" bundle=/var/run/docker/containerd/9dd2b629320270c4d84df5d85e0bb0930deb3e6e8b63d933ceec3120bdbc682d module=libcontainerd namespace=moby root=/var/lib/docker/overlay2/825f56dc0f463eee0313d20940511d627f9193b70fbe42fe94d5970efe3a1566/merged
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22-06:00" level=debug msg="event published" module="containerd/containers" ns=moby topic="/containers/create" type=containerd.events.ContainerCreate
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22-06:00" level=info msg="shim docker-containerd-shim started" address="/containerd-shim/moby/9dd2b629320270c4d84df5d85e0bb0930deb3e6e8b63d933ceec3120bdbc682d/shim.sock" debug=true module="containerd/tasks" pid=20421
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22-06:00" level=debug msg="registering ttrpc server"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22-06:00" level=debug msg="serving api on unix socket" socket="[inherited from
parent]"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.651257657-06:00" level=debug msg="sandbox set key processing took 59.022624ms for container 9dd2b629320270c4d84df5d85e0bb0930deb3e6e8b63d933ceec3120bdbc682d"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22-06:00" level=debug msg="event published" module="containerd/tasks" ns=moby topic="/tasks/create" type=containerd.events.TaskCreate
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.693475256-06:00" level=debug msg=event module=libcontainerd namespace=moby topic=/tasks/create
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22-06:00" level=debug msg="event published" module="containerd/tasks" ns=moby topic="/tasks/start" type=containerd.events.TaskStart
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.728065958-06:00" level=debug msg=event module=libcontainerd namespace=moby topic=/tasks/start
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22-06:00" level=debug msg="event published" module="containerd/events" ns=moby
topic="/tasks/exit" type=containerd.events.TaskExit
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.742053989-06:00" level=debug msg=event module=libcontainerd namespace=moby topic=/tasks/exit
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.746510518-06:00" level=debug msg="Calling POST /v1.38/containers/9dd2b629320270c4d84df5d85e0bb0930deb3e6e8b63d933ceec3120bdbc682d/resize?h=64&w=148"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22-06:00" level=info msg="shim reaped" id=9dd2b629320270c4d84df5d85e0bb0930deb3e6e8b63d933ceec3120bdbc682d module="containerd/tasks"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22-06:00" level=debug msg="event published" module="containerd/tasks" ns=moby topic="/tasks/delete" type=containerd.events.TaskDelete
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.776549159-06:00" level=debug msg=event module=libcontainerd namespace=moby topic=/tasks/delete
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.776569439-06:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.776599711-06:00" level=debug msg="attach: stderr: end"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.776604874-06:00" level=debug msg="attach: stdout: end"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.776617674-06:00" level=debug msg="attach: stdin: end"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.776628037-06:00" level=debug msg="attach done"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.776674611-06:00" level=debug msg="Closing buffered stdin pipe"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.776825113-06:00" level=debug msg="Revoking external connectivity on endpoint tender_goldstine (82cd17f08207141d15fd9570403c400e540340894d1dad5d5f2d871076c3d6d1)"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.777630240-06:00" level=debug msg="DeleteConntrackEntries purged ipv4:1, ipv6:0"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.844137388-06:00" level=debug msg="Releasing addresses for endpoint tender_goldstine's interface on network bridge"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.844158649-06:00" level=debug msg="ReleaseAddress(LocalDefault/172.17.0.0/16, 172.17.0.2)"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22.844176525-06:00" level=debug msg="Released address PoolID:LocalDefault/172.17.0.0/16, Address:172.17.0.2 Sequence:App: ipam/default/data, ID: LocalDefault/172.17.0.0/16, DBIndex: 0x0, Bits: 65536, Unselected: 65532, Sequence: (0xe0000000, 1)->(0x0, 2046)->(0x1, 1)->end Curr:3"
Feb 08 09:12:22 mark dockerd[20057]: time="2019-02-08T09:12:22-06:00" level=debug msg="event published" module="containerd/containers" ns=moby topic="/containers/delete" type=containerd.events.ContainerDelete
mrkz commented 5 years ago

Also, there's 2 updates applied to the container: Jump to 27210 and then jumps to 27670

I've updated to 27210, created a commit on such update and the error is not reproducible, however I later update to 27670 release, created a commit and the error appears.

The changed files between 27210..27670 according to swupd verify:

root@b816258ca5d9 / # swupd check-update
Current OS version: 27210
There is a new OS version available: 27670
root@b816258ca5d9 / # swupd verify -m 27670 --fix --force
Verifying version 27670
WARNING: the force or picky option is specified; ignoring version mismatch for verify --fix
Verifying files
        ...100%
Starting download of remaining update content. This may take a while...
        ...100%
Finishing download of update content...
Adding any missing files

Missing file: /usr/bin/3rd-party-post

        fixed
        ...0%
Missing file: /usr/bin/swupd-3rd-party

        fixed
        ...9%
Missing file: /usr/lib64/haswell/libm-2.29.so

        fixed
        ...26%
Missing file: /usr/lib64/ld-2.29.so

        fixed
        ...27%
Missing file: /usr/lib64/libBrokenLocale-2.29.so

        fixed

Missing file: /usr/lib64/libanl-2.29.so

        fixed

Missing file: /usr/lib64/libc-2.29.so

        fixed

Missing file: /usr/lib64/libcrypt-2.29.so

        fixed
        ...28%
Missing file: /usr/lib64/libdl-2.29.so

        fixed

Missing file: /usr/lib64/libgio-2.0.so.0.5800.3

        fixed
        ...29%
Missing file: /usr/lib64/libglib-2.0.so.0.5800.3

        fixed

Missing file: /usr/lib64/libgmodule-2.0.so.0.5800.3

        fixed

Missing file: /usr/lib64/libgobject-2.0.so.0.5800.3

        fixed

Missing file: /usr/lib64/libgthread-2.0.so.0.5800.3

        fixed

Missing file: /usr/lib64/libm-2.29.so

        fixed
        ...30%
Missing file: /usr/lib64/libmvec-2.29.so

        fixed

Missing file: /usr/lib64/libnsl-2.29.so

        fixed

Missing file: /usr/lib64/libnss_compat-2.29.so

        fixed

Missing file: /usr/lib64/libnss_dns-2.29.so

        fixed
        ...31%
Missing file: /usr/lib64/libnss_files-2.29.so

        fixed

Missing file: /usr/lib64/libnss_hesiod-2.29.so

        fixed

Missing file: /usr/lib64/libpthread-2.29.so

        fixed

Missing file: /usr/lib64/libresolv-2.29.so

        fixed
        ...32%
Missing file: /usr/lib64/librt-2.29.so

        fixed

Missing file: /usr/lib64/libutil-2.29.so

        fixed

Missing file: /usr/share/ca-certs/trusted/Certigna_Root_CA.crt

        fixed
        ...47%
Missing file: /usr/share/ca-certs/trusted/GTS_Root_R1.crt

        fixed
        ...48%
Missing file: /usr/share/ca-certs/trusted/GTS_Root_R2.crt

        fixed

Missing file: /usr/share/ca-certs/trusted/GTS_Root_R3.crt

        fixed

Missing file: /usr/share/ca-certs/trusted/GTS_Root_R4.crt

        fixed

Missing file: /usr/share/ca-certs/trusted/UCA_Extended_Validation_Root.crt

        fixed
        ...50%
Missing file: /usr/share/ca-certs/trusted/UCA_Global_G2_Root.crt

        fixed

Missing file: /usr/share/clear/bundles/os-core-webproxy

        fixed

Missing file: /usr/share/clr-bundles/aws-greengrass-iot

        fixed
        ...51%
Missing file: /usr/share/clr-bundles/desktop-kde-games

        fixed
        ...52%
Missing file: /usr/share/clr-bundles/dev-utils-gui

        fixed

Missing file: /usr/share/clr-bundles/devpkg-curl

        fixed

Missing file: /usr/share/clr-bundles/devpkg-fuse

        fixed

Missing file: /usr/share/clr-bundles/devpkg-glm

        fixed

Missing file: /usr/share/clr-bundles/devpkg-libxslt

        fixed
        ...53%
Missing file: /usr/share/clr-bundles/devpkg-ncurses

        fixed

Missing file: /usr/share/clr-bundles/devpkg-opencv

        fixed

Missing file: /usr/share/clr-bundles/devpkg-openmpi

        fixed

Missing file: /usr/share/clr-bundles/devpkg-openssl

        fixed

Missing file: /usr/share/clr-bundles/devpkg-systemd

        fixed
        ...54%
Missing file: /usr/share/clr-bundles/devpkg-xcb-proto

        fixed

Missing file: /usr/share/clr-bundles/fonts-basic

        fixed

Missing file: /usr/share/clr-bundles/hardware-printing

        fixed

Missing file: /usr/share/clr-bundles/kde-frameworks5

        fixed
        ...55%
Missing file: /usr/share/clr-bundles/kdiff3

        fixed

Missing file: /usr/share/clr-bundles/kernel-native-dkms

        fixed

Missing file: /usr/share/clr-bundles/machine-learning-pytorch

        fixed

Missing file: /usr/share/clr-bundles/maker-3dprinting

        fixed

Missing file: /usr/share/clr-bundles/ocaml-basic

        fixed
        ...56%
Missing file: /usr/share/clr-bundles/os-core-webproxy

        fixed

Missing file: /usr/share/clr-bundles/python-testing

        fixed
        ...57%
Missing file: /usr/share/gdb/auto-load/usr/lib32/libglib-2.0.so.0.5800.3-gdb.py

        fixed
        ...61%
Missing file: /usr/share/gdb/auto-load/usr/lib32/libgobject-2.0.so.0.5800.3-gdb.py

        fixed

Missing file: /usr/share/gdb/auto-load/usr/lib64/libglib-2.0.so.0.5800.3-gdb.py

        fixed

Missing file: /usr/share/gdb/auto-load/usr/lib64/libgobject-2.0.so.0.5800.3-gdb.py

        fixed

Missing file: /usr/share/man/man1/3rd-party-post.1

        fixed
        ...66%
Missing file: /usr/share/man/man1/swupd-3rd-party.1

        fixed
        ...71%
Missing file: /usr/share/man/man7/swupd-alias.7

        fixed
        ...75%
Missing file: /usr/share/package-licenses/clr-user-bundles

        fixed
        ...85%
Missing file: /usr/share/package-licenses/clr-user-bundles/COPYING

        fixed

Missing file: /usr/share/package-licenses/clr-user-bundles/vendor_github.com_BurntSushi_toml_COPYING

        fixed

Missing file: /usr/share/package-licenses/clr-user-bundles/vendor_github.com_inconshreveable_mousetrap_LICENSE

        fixed

Missing file: /usr/share/package-licenses/clr-user-bundles/vendor_github.com_spf13_cobra_LICENSE.txt

        fixed

Missing file: /usr/share/package-licenses/clr-user-bundles/vendor_github.com_spf13_pflag_LICENSE

        fixed

Missing file: /usr/share/package-licenses/pacrunner

        fixed
        ...87%
Missing file: /usr/share/package-licenses/pacrunner/COPYING

        fixed

Missing file: /usr/share/package-licenses/pacrunner/COPYING.LIB

        fixed

Missing file: /usr/share/polkit-1/actions/org.clearlinux.swupd.policy

        fixed
        ...89%
Missing file: /usr/share/polkit-1/rules.d

        fixed

Missing file: /usr/share/polkit-1/rules.d/org.clearlinux.swupd.rules

        fixed
        ...100%
Fixing modified files
        ...0%
Hash mismatch for file: /usr/bin/bootctl
        fixed

Hash mismatch for file: /usr/bin/btrfs
        fixed

Hash mismatch for file: /usr/bin/btrfs-convert
        fixed

Hash mismatch for file: /usr/bin/btrfs-find-root
        fixed

Hash mismatch for file: /usr/bin/btrfs-image
        fixed

Hash mismatch for file: /usr/bin/btrfs-map-logical
        fixed
        ...1%
Hash mismatch for file: /usr/bin/btrfs-select-super
        fixed

Hash mismatch for file: /usr/bin/btrfstune
        fixed

Hash mismatch for file: /usr/bin/busctl
        fixed

Hash mismatch for file: /usr/bin/clr-boot-manager
        fixed

Hash mismatch for file: /usr/bin/clr_power
        fixed
        ...2%
Hash mismatch for file: /usr/bin/coredumpctl
        fixed
        ...3%
Hash mismatch for file: /usr/bin/gencat
        fixed

Hash mismatch for file: /usr/bin/getconf
        fixed

Hash mismatch for file: /usr/bin/getent
        fixed
        ...4%
Hash mismatch for file: /usr/bin/hostnamectl
        fixed

Hash mismatch for file: /usr/bin/iconv
        fixed

Hash mismatch for file: /usr/bin/iconvconfig
        fixed

Hash mismatch for file: /usr/bin/journalctl
        fixed

Hash mismatch for file: /usr/bin/ldconfig
        fixed
        ...5%
Hash mismatch for file: /usr/bin/locale
        fixed

Hash mismatch for file: /usr/bin/localectl
        fixed

Hash mismatch for file: /usr/bin/loginctl
        fixed

Hash mismatch for file: /usr/bin/machinectl
        fixed

Hash mismatch for file: /usr/bin/manual-proxy-test
        fixed
        ...6%
Hash mismatch for file: /usr/bin/mkfs.btrfs
        fixed

Hash mismatch for file: /usr/bin/networkctl
        fixed

Hash mismatch for file: /usr/bin/openssl
        fixed

Hash mismatch for file: /usr/bin/pacdiscovery
        fixed

Hash mismatch for file: /usr/bin/pacrunner
        fixed
        ...7%
Hash mismatch for file: /usr/bin/pcprofiledump
        fixed

Hash mismatch for file: /usr/bin/pldd
        fixed

Hash mismatch for file: /usr/bin/proxy
        fixed

Hash mismatch for file: /usr/bin/resolvectl
        fixed
        ...8%
Hash mismatch for file: /usr/bin/rpcgen
        fixed

Hash mismatch for file: /usr/bin/sotruss
        fixed

Hash mismatch for file: /usr/bin/sprof
        fixed
        ...9%
Hash mismatch for file: /usr/bin/swupd
        fixed

Hash mismatch for file: /usr/bin/systemctl
        fixed

Hash mismatch for file: /usr/bin/systemd-analyze
        fixed

Hash mismatch for file: /usr/bin/systemd-ask-password
        fixed

Hash mismatch for file: /usr/bin/systemd-cat
        fixed

Hash mismatch for file: /usr/bin/systemd-cgls
        fixed

Hash mismatch for file: /usr/bin/systemd-cgtop
        fixed

Hash mismatch for file: /usr/bin/systemd-delta
        fixed

Hash mismatch for file: /usr/bin/systemd-detect-virt
        fixed

Hash mismatch for file: /usr/bin/systemd-escape
        fixed

Hash mismatch for file: /usr/bin/systemd-inhibit
        fixed

Hash mismatch for file: /usr/bin/systemd-machine-id-setup
        fixed

Hash mismatch for file: /usr/bin/systemd-mount
        fixed

Hash mismatch for file: /usr/bin/systemd-notify
        fixed

Hash mismatch for file: /usr/bin/systemd-nspawn
        fixed

Hash mismatch for file: /usr/bin/systemd-path
        fixed

Hash mismatch for file: /usr/bin/systemd-run
        fixed

Hash mismatch for file: /usr/bin/systemd-socket-activate
        fixed

Hash mismatch for file: /usr/bin/systemd-stdio-bridge
        fixed

Hash mismatch for file: /usr/bin/systemd-tmpfiles
        fixed

Hash mismatch for file: /usr/bin/systemd-tty-ask-password-agent
        fixed
        ...10%
Hash mismatch for file: /usr/bin/timedatectl
        fixed

Hash mismatch for file: /usr/bin/tzselect
        fixed

Hash mismatch for file: /usr/bin/udevadm
        fixed
        ...11%
Hash mismatch for file: /usr/bin/verifytime
        fixed

Hash mismatch for file: /usr/bin/xtrace
        fixed

Hash mismatch for file: /usr/bin/zdump
        fixed

Hash mismatch for file: /usr/bin/zic
        fixed
        ...12%
Hash mismatch for file: /usr/lib/os-release
        fixed

Hash mismatch for file: /usr/lib/systemd/boot/efi/linuxx64.efi.stub
        fixed

Hash mismatch for file: /usr/lib/systemd/boot/efi/systemd-bootx64.efi
        fixed

Hash mismatch for file: /usr/lib/systemd/libsystemd-shared-239.so
        fixed
        ...13%
Hash mismatch for file: /usr/lib/systemd/portablectl
        fixed

Hash mismatch for file: /usr/lib/systemd/system-generators/systemd-cryptsetup-generator
        fixed

Hash mismatch for file: /usr/lib/systemd/system-generators/systemd-debug-generator
        fixed

Hash mismatch for file: /usr/lib/systemd/system-generators/systemd-fstab-generator
        fixed

Hash mismatch for file: /usr/lib/systemd/system-generators/systemd-getty-generator
        fixed

Hash mismatch for file: /usr/lib/systemd/system-generators/systemd-gpt-auto-generator
        fixed
        ...16%
Hash mismatch for file: /usr/lib/systemd/system/pacdiscovery.service
        fixed
        ...19%
Hash mismatch for file: /usr/lib/systemd/systemd
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-ac-power
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-backlight
        fixed
        ...20%
Hash mismatch for file: /usr/lib/systemd/systemd-binfmt
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-bootchart
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-cgroups-agent
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-coredump
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-cryptsetup
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-dissect
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-export
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-fsck
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-growfs
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-hibernate-resume
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-hostnamed
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-initctl
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-journald
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-localed
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-logind
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-machined
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-makefs
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-modules-load
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-networkd
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-networkd-wait-online
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-portabled
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-quotacheck
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-random-seed
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-remount-fs
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-reply-password
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-resolved
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-rfkill
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-shutdown
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-sleep
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-socket-proxyd
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-sulogin-shell
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-sysctl
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-time-wait-sync
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-timedated
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-timesyncd
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-udevd
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-update-utmp
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-user-runtime-dir
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-user-sessions
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-vconsole-setup
        fixed
        ...21%
Hash mismatch for file: /usr/lib/systemd/systemd-veritysetup
        fixed

Hash mismatch for file: /usr/lib/systemd/systemd-volatile-root
        fixed

Hash mismatch for file: /usr/lib/systemd/user-environment-generators/30-systemd-environment-d-generator
        fixed

Hash mismatch for file: /usr/lib/sysusers.d/clear.conf
        fixed
        ...22%
Hash mismatch for file: /usr/lib/udev/ata_id
        fixed
        ...23%
Hash mismatch for file: /usr/lib/udev/scsi_id
        fixed

Hash mismatch for file: /usr/lib64/audit/sotruss-lib.so
        fixed

Hash mismatch for file: /usr/lib64/elfutils/libebl_aarch64-0.175.so
        fixed

Hash mismatch for file: /usr/lib64/elfutils/libebl_bpf-0.175.so
        fixed

Hash mismatch for file: /usr/lib64/elfutils/libebl_i386-0.175.so
        fixed

Hash mismatch for file: /usr/lib64/elfutils/libebl_riscv-0.175.so
        fixed

Hash mismatch for file: /usr/lib64/elfutils/libebl_x86_64-0.175.so
        fixed

Hash mismatch for file: /usr/lib64/engines-1.1/afalg.so
        fixed

Hash mismatch for file: /usr/lib64/engines-1.1/capi.so
        fixed

Hash mismatch for file: /usr/lib64/engines-1.1/padlock.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ANSI_X3.110.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ARMSCII-8.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ASMO_449.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/BIG5.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/BIG5HKSCS.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/BRF.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP10007.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1125.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1250.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1251.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1252.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1253.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1254.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1255.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1256.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1257.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1258.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP737.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP770.so
        fixed
        ...24%
Hash mismatch for file: /usr/lib64/gconv/CP771.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP772.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP773.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP774.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP775.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP932.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CSN_369103.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CWI.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/EUC-CN.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/EUC-JISX0213.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/EUC-JP-MS.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/EUC-JP.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/EUC-KR.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/EUC-TW.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/GEORGIAN-ACADEMY.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/GEORGIAN-PS.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/GOST_19768-74.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/GREEK-CCITT.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/GREEK7-OLD.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/GREEK7.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/HP-GREEK8.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/HP-ROMAN8.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/HP-ROMAN9.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/HP-THAI8.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/HP-TURKISH8.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/IBM858.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/IEC_P27-1.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/INIS-8.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/INIS-CYRILLIC.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/INIS.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISIRI-3342.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO-2022-CN-EXT.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO-2022-CN.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO-2022-JP-3.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO-2022-JP.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO-2022-KR.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO-IR-197.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO-IR-209.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO646.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-1.so
        fixed
        ...25%
Hash mismatch for file: /usr/lib64/gconv/ISO8859-10.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-11.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-13.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-14.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-15.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-16.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-2.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-3.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-4.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-5.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-6.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-7.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-8.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-9.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-9E.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO_10367-BOX.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO_11548-1.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO_2033.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO_5427-EXT.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO_5427.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO_5428.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO_6937-2.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO_6937.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/JOHAB.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/KOI-8.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/KOI8-R.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/KOI8-RU.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/KOI8-T.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/KOI8-U.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/LATIN-GREEK-1.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/LATIN-GREEK.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/MAC-CENTRALEUROPE.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/MAC-IS.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/MAC-SAMI.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/MAC-UK.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/MACINTOSH.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/MIK.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/NATS-DANO.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/NATS-SEFI.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/PT154.so
        fixed
        ...26%
Hash mismatch for file: /usr/lib64/gconv/RK1048.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/SAMI-WS2.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/SHIFT_JISX0213.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/SJIS.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/T.61.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/TCVN5712-1.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/TIS-620.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/TSCII.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/UHC.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/UNICODE.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/UTF-16.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/UTF-32.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/UTF-7.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/VISCII.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/gconv-modules
        fixed

Hash mismatch for file: /usr/lib64/gconv/libCNS.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/libGB.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/libISOIR165.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/libJIS.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/libJISX0213.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/libKSC.so
        fixed

Hash mismatch for file: /usr/lib64/glibc/getconf/POSIX_V6_LP64_OFF64
        fixed

Hash mismatch for file: /usr/lib64/glibc/getconf/POSIX_V7_LP64_OFF64
        fixed

Hash mismatch for file: /usr/lib64/glibc/getconf/XBS5_LP64_OFF64
        fixed

Hash mismatch for file: /usr/lib64/haswell/libarchive.so.13.3.3
        fixed

Hash mismatch for file: /usr/lib64/haswell/libm.so.6
        fixed
        ...27%
Hash mismatch for file: /usr/lib64/haswell/libz.so.1.2.11.1-motley
        fixed

Hash mismatch for file: /usr/lib64/ld-linux-x86-64.so.2
        fixed

Hash mismatch for file: /usr/lib64/libBrokenLocale.so.1
        fixed

Hash mismatch for file: /usr/lib64/libSegFault.so
        fixed

Hash mismatch for file: /usr/lib64/libanl.so.1
        fixed

Hash mismatch for file: /usr/lib64/libarchive.so.13.3.3
        fixed

Hash mismatch for file: /usr/lib64/libasm-0.175.so
        fixed

Hash mismatch for file: /usr/lib64/libbtrfs.so.0.1
        fixed

Hash mismatch for file: /usr/lib64/libbtrfsutil.so.1.1.0
        fixed

Hash mismatch for file: /usr/lib64/libc.so.6
        fixed
        ...28%
Hash mismatch for file: /usr/lib64/libcrypt.so.1
        fixed

Hash mismatch for file: /usr/lib64/libcrypto.so.1.1
        fixed

Hash mismatch for file: /usr/lib64/libcurl.so.4.5.0
        fixed

Hash mismatch for file: /usr/lib64/libdl.so.2
        fixed

Hash mismatch for file: /usr/lib64/libdw-0.175.so
        fixed

Hash mismatch for file: /usr/lib64/libelf-0.175.so
        fixed

Hash mismatch for file: /usr/lib64/libform.so.6.1
        fixed
        ...29%
Hash mismatch for file: /usr/lib64/libformw.so.6.1
        fixed

Hash mismatch for file: /usr/lib64/libgcc_s.so.1
        fixed

Hash mismatch for file: /usr/lib64/libgio-2.0.so.0
        fixed

Hash mismatch for file: /usr/lib64/libglib-2.0.so.0
        fixed

Hash mismatch for file: /usr/lib64/libgmodule-2.0.so.0
        fixed

Hash mismatch for file: /usr/lib64/libgobject-2.0.so.0
        fixed

Hash mismatch for file: /usr/lib64/libgthread-2.0.so.0
        fixed
        ...30%
Hash mismatch for file: /usr/lib64/libm.so.6
        fixed

Hash mismatch for file: /usr/lib64/libmemusage.so
        fixed

Hash mismatch for file: /usr/lib64/libmenu.so.6.1
        fixed

Hash mismatch for file: /usr/lib64/libmenuw.so.6.1
        fixed

Hash mismatch for file: /usr/lib64/libmvec.so.1
        fixed

Hash mismatch for file: /usr/lib64/libncurses.so.6.1
        fixed

Hash mismatch for file: /usr/lib64/libncursesw.so.6.1
        fixed

Hash mismatch for file: /usr/lib64/libnghttp2.so.14.17.1
        fixed

Hash mismatch for file: /usr/lib64/libnsl.so.1
        fixed

Hash mismatch for file: /usr/lib64/libnss_compat.so.2
        fixed
        ...31%
Hash mismatch for file: /usr/lib64/libnss_dns.so.2
        fixed

Hash mismatch for file: /usr/lib64/libnss_files.so.2
        fixed

Hash mismatch for file: /usr/lib64/libnss_hesiod.so.2
        fixed

Hash mismatch for file: /usr/lib64/libnss_myhostname.so.2
        fixed

Hash mismatch for file: /usr/lib64/libnss_mymachines.so.2
        fixed

Hash mismatch for file: /usr/lib64/libnss_resolve.so.2
        fixed

Hash mismatch for file: /usr/lib64/libnss_systemd.so.2
        fixed

Hash mismatch for file: /usr/lib64/libpanel.so.6.1
        fixed

Hash mismatch for file: /usr/lib64/libpanelw.so.6.1
        fixed

Hash mismatch for file: /usr/lib64/libpcprofile.so
        fixed

Hash mismatch for file: /usr/lib64/libproxy.so.1.0.0
        fixed

Hash mismatch for file: /usr/lib64/libpthread.so.0
        fixed
        ...32%
Hash mismatch for file: /usr/lib64/libresolv.so.2
        fixed

Hash mismatch for file: /usr/lib64/librt.so.1
        fixed

Hash mismatch for file: /usr/lib64/libssl.so.1.1
        fixed

Hash mismatch for file: /usr/lib64/libsystemd.so.0.23.0
        fixed

Hash mismatch for file: /usr/lib64/libthread_db-1.0.so
        fixed

Hash mismatch for file: /usr/lib64/libtinfo.so.6.1
        fixed

Hash mismatch for file: /usr/lib64/libtinfow.so.6.1
        fixed

Hash mismatch for file: /usr/lib64/libudev.so.1.6.11
        fixed

Hash mismatch for file: /usr/lib64/libutil.so.1
        fixed

Hash mismatch for file: /usr/lib64/libz.so.1.2.11.1-motley
        fixed
        ...33%
Hash mismatch for file: /usr/lib64/security/pam_systemd.so
        fixed
        ...37%
Hash mismatch for file: /usr/libexec/glib-compile-schemas
        fixed
        ...50%
Hash mismatch for file: /usr/share/clear/commandlist.csv
        fixed
        ...51%
Hash mismatch for file: /usr/share/clear/version
        fixed

Hash mismatch for file: /usr/share/clear/versionstamp
        fixed

Hash mismatch for file: /usr/share/clr-bundles/ansible
        fixed

Hash mismatch for file: /usr/share/clr-bundles/c-basic
        fixed

Hash mismatch for file: /usr/share/clr-bundles/c-basic-legacy
        fixed

Hash mismatch for file: /usr/share/clr-bundles/cloud-api
        fixed

Hash mismatch for file: /usr/share/clr-bundles/cloud-native-basic
        fixed

Hash mismatch for file: /usr/share/clr-bundles/computer-vision-basic
        fixed

Hash mismatch for file: /usr/share/clr-bundles/containers-basic-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/containers-virt-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/cryptoprocessor-management-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/database-basic-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/desktop
        fixed

Hash mismatch for file: /usr/share/clr-bundles/desktop-apps-extras
        fixed

Hash mismatch for file: /usr/share/clr-bundles/desktop-assets
        fixed
        ...52%
Hash mismatch for file: /usr/share/clr-bundles/desktop-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/desktop-kde
        fixed

Hash mismatch for file: /usr/share/clr-bundles/desktop-kde-apps
        fixed

Hash mismatch for file: /usr/share/clr-bundles/desktop-kde-libs
        fixed

Hash mismatch for file: /usr/share/clr-bundles/dev-utils-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-at-spi2-atk
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-at-spi2-core
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-atk
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-cairo
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-dbus
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-evolution
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-evolution-data-server
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-expat
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-fontconfig
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-freetype
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-fribidi
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-gdk-pixbuf
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-glib
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-gnome-desktop
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-graphite
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-gsettings-desktop-schemas
        fixed
        ...53%
Hash mismatch for file: /usr/share/clr-bundles/devpkg-gtk3
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-harfbuzz
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-icu4c
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-libX11
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-libdrm
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-libepoxy
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-libical
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-libpng
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-libpthread-stubs
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-libsecret
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-libsoup
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-libxkbcommon
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-libxml2
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-mesa
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-nspr
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-nss
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-pango
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-pcre
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-pixman
        fixed
        ...54%
Hash mismatch for file: /usr/share/clr-bundles/devpkg-qtbase
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-sqlite-autoconf
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-util-linux
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-wayland
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-wayland-protocols
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-webkitgtk
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-xorgproto
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-xz
        fixed

Hash mismatch for file: /usr/share/clr-bundles/devpkg-zlib
        fixed

Hash mismatch for file: /usr/share/clr-bundles/domotica
        fixed

Hash mismatch for file: /usr/share/clr-bundles/editors-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/education-primary
        fixed

Hash mismatch for file: /usr/share/clr-bundles/education-secondary
        fixed

Hash mismatch for file: /usr/share/clr-bundles/games
        fixed

Hash mismatch for file: /usr/share/clr-bundles/games-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/go-basic-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/haskell-basic
        fixed
        ...55%
Hash mismatch for file: /usr/share/clr-bundles/kvm-host-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/lua-basic
        fixed

Hash mismatch for file: /usr/share/clr-bundles/machine-learning-basic
        fixed

Hash mismatch for file: /usr/share/clr-bundles/mail-utils
        fixed

Hash mismatch for file: /usr/share/clr-bundles/mail-utils-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/mercurial
        fixed

Hash mismatch for file: /usr/share/clr-bundles/mixer
        fixed
        ...56%
Hash mismatch for file: /usr/share/clr-bundles/network-basic-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/os-clr-on-clr
        fixed

Hash mismatch for file: /usr/share/clr-bundles/os-clr-on-clr-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/os-core-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/os-core-update
        fixed

Hash mismatch for file: /usr/share/clr-bundles/os-core-update-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/os-testsuite
        fixed

Hash mismatch for file: /usr/share/clr-bundles/os-testsuite-graphics
        fixed

Hash mismatch for file: /usr/share/clr-bundles/os-testsuite-phoronix
        fixed

Hash mismatch for file: /usr/share/clr-bundles/os-testsuite-phoronix-desktop
        fixed

Hash mismatch for file: /usr/share/clr-bundles/os-testsuite-phoronix-server
        fixed

Hash mismatch for file: /usr/share/clr-bundles/os-utils-gui
        fixed

Hash mismatch for file: /usr/share/clr-bundles/os-utils-gui-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/package-builder
        fixed

Hash mismatch for file: /usr/share/clr-bundles/packages
        fixed

Hash mismatch for file: /usr/share/clr-bundles/performance-tools
        fixed

Hash mismatch for file: /usr/share/clr-bundles/perl-basic
        fixed

Hash mismatch for file: /usr/share/clr-bundles/perl-basic-dev
        fixed
        ...57%
Hash mismatch for file: /usr/share/clr-bundles/python-basic-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/python-extras
        fixed

Hash mismatch for file: /usr/share/clr-bundles/qt-basic
        fixed

Hash mismatch for file: /usr/share/clr-bundles/qt-basic-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/scm-server
        fixed

Hash mismatch for file: /usr/share/clr-bundles/service-os
        fixed

Hash mismatch for file: /usr/share/clr-bundles/service-os-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/software-defined-cockpit
        fixed

Hash mismatch for file: /usr/share/clr-bundles/software-defined-cockpit-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/storage-utils-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/sysadmin-basic-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/sysadmin-hostmgmt
        fixed

Hash mismatch for file: /usr/share/clr-bundles/user-basic-dev
        fixed
        ...58%
Hash mismatch for file: /usr/share/clr-bundles/vim
        fixed

Hash mismatch for file: /usr/share/clr-bundles/x11-tools
        fixed

Hash mismatch for file: /usr/share/clr-bundles/xfce4-desktop
        fixed
        ...59%
Hash mismatch for file: /usr/share/defaults/etc/passwd
        fixed

Hash mismatch for file: /usr/share/defaults/etc/profile
        fixed

Hash mismatch for file: /usr/share/defaults/etc/profile.d/10-command-not-found.sh
        fixed

Hash mismatch for file: /usr/share/defaults/etc/shadow
        fixed
        ...61%
Hash mismatch for file: /usr/share/glib-2.0/codegen/config.py
        fixed

Hash mismatch for file: /usr/share/hwdata/pci.ids
        fixed

Hash mismatch for file: /usr/share/hwdata/usb.ids
        fixed
        ...63%
Hash mismatch for file: /usr/share/locale/en_US.UTF-8/LC_TIME
        fixed
        ...67%
Hash mismatch for file: /usr/share/man/man1/clr-boot-manager.1
        fixed
        ...71%
Hash mismatch for file: /usr/share/man/man1/swupd.1
        fixed
        ...99%
Hash mismatch for file: /usr/share/zoneinfo/zone.tab
        fixed
        ...100%
File that should be deleted: /usr/share/package-licenses/curl/COPYING
        deleted
File that should be deleted: /usr/share/package-licenses/curl
        deleted
File that should be deleted: /usr/share/gdb/auto-load/usr/lib64/libgobject-2.0.so.0.5800.2-gdb.py
        deleted
File that should be deleted: /usr/share/gdb/auto-load/usr/lib64/libglib-2.0.so.0.5800.2-gdb.py
        deleted
File that should be deleted: /usr/share/gdb/auto-load/usr/lib32/libgobject-2.0.so.0.5800.2-gdb.py
        deleted
File that should be deleted: /usr/share/gdb/auto-load/usr/lib32/libglib-2.0.so.0.5800.2-gdb.py
        deleted
File that should be deleted: /usr/share/doc/pacrunner/COPYING.LIB
        deleted
File that should be deleted: /usr/share/doc/pacrunner/COPYING
        deleted
File that should be deleted: /usr/share/doc/pacrunner
        deleted
File that should be deleted: /usr/share/ca-certs/trusted/Visa_eCommerce_Root.crt
        deleted
File that should be deleted: /usr/share/ca-certs/trusted/OpenTrust_Root_CA_G3.crt
        deleted
File that should be deleted: /usr/share/ca-certs/trusted/OpenTrust_Root_CA_G2.crt
        deleted
File that should be deleted: /usr/share/ca-certs/trusted/OpenTrust_Root_CA_G1.crt
        deleted
File that should be deleted: /usr/share/ca-certs/trusted/Certplus_Root_CA_G2.crt
        deleted
File that should be deleted: /usr/share/ca-certs/trusted/Certplus_Root_CA_G1.crt
        deleted
File that should be deleted: /usr/share/ca-certs/trusted/AC_Raíz_Certicámara_S.A..crt
        deleted
File that should be deleted: /usr/lib64/libutil-2.28.so
        deleted
File that should be deleted: /usr/lib64/librt-2.28.so
        deleted
File that should be deleted: /usr/lib64/libresolv-2.28.so
        deleted
File that should be deleted: /usr/lib64/libpthread-2.28.so
        deleted
File that should be deleted: /usr/lib64/libnss_hesiod-2.28.so
        deleted
File that should be deleted: /usr/lib64/libnss_files-2.28.so
        deleted
File that should be deleted: /usr/lib64/libnss_dns-2.28.so
        deleted
File that should be deleted: /usr/lib64/libnss_compat-2.28.so
        deleted
File that should be deleted: /usr/lib64/libnsl-2.28.so
        deleted
File that should be deleted: /usr/lib64/libmvec-2.28.so
        deleted
File that should be deleted: /usr/lib64/libm-2.28.so
        deleted
File that should be deleted: /usr/lib64/libgthread-2.0.so.0.5800.2
        deleted
File that should be deleted: /usr/lib64/libgobject-2.0.so.0.5800.2
        deleted
File that should be deleted: /usr/lib64/libgmodule-2.0.so.0.5800.2
        deleted
File that should be deleted: /usr/lib64/libglib-2.0.so.0.5800.2
        deleted
File that should be deleted: /usr/lib64/libgio-2.0.so.0.5800.2
        deleted
File that should be deleted: /usr/lib64/libdl-2.28.so
        deleted
File that should be deleted: /usr/lib64/libcrypt-2.28.so
        deleted
File that should be deleted: /usr/lib64/libc-2.28.so
        deleted
File that should be deleted: /usr/lib64/libanl-2.28.so
        deleted
File that should be deleted: /usr/lib64/libBrokenLocale-2.28.so
        deleted
File that should be deleted: /usr/lib64/ld-2.28.so
        deleted
File that should be deleted: /usr/lib64/haswell/libm-2.28.so
        deleted
Inspected 3994 files
  75 files were missing
    75 of 75 missing files were replaced
    0 of 75 missing files were not replaced
  442 files did not match
    442 of 442 files were fixed
    0 of 442 files were not fixed
  39 files found which should be deleted
    39 of 39 files were deleted
    0 of 39 files were not deleted
Calling post-update helper scripts.
none
[ERROR] cbm (../src/lib/files.c:L165): Invalid block device: /
[ERROR] cbm (../src/lib/system_stub.c:L31): Invalid block device: 0:52
Out of memoryFix successful
mrkz commented 5 years ago

I performed a swupd bisect (manually :grimacing: ) between 27210 and 27670, creating new images on each jump, the first failing update results to be 27600. The release notes of such release can be found on https://download.clearlinux.org/releases/27600/clear/RELEASENOTES

Here's the content of RELEASENOTES:

Release notes for the update from 27590 to 27600

Updated packages:
     R-fGarch                            3042.83-5       -> 3042.83.1-6    
     R-pamr                              1.55-12         -> 1.56-13        
     R-partykit                          1.2.2-15        -> 1.2.3-16       
     Shapely                             1.6.4.post2-1   -> 1.6.4.post2-2  
     acrn-hypervisor                     2019w05.2.140000p-158 -> 2019w05.4.140000p-159
     atomicwrites                        1.2.1-21        -> 1.3.0-22       
     certbot                             0.30.2-42       -> 0.30.2-44      
     clr-bundles                         27530-34        -> 27580-35       
     cnf                                 1-135           -> 1-136          
     colcon-core                         0.3.15-14       -> 0.3.15-15      
     dask                                1.1.0-50        -> 1.1.1-51       
     gcc                                 8.2.0-291       -> 8.2.0-292      
     gensim                              3.7.0-34        -> 3.7.1-35       
     glibc                               2.28-238        -> 2.29-238       
     luarocks                            3.0.4-4         -> 3.0.4-5        
     optipng                             0.7.7-1         -> 0.7.7-2        
     padatious                           0.4.5-2         -> 0.4.6-3        
     python-editor                       1.0.3-21        -> 1.0.4-22       
     scikit-learn                        0.20.2-68       -> 0.20.2-69      
     spyrk                               0.0.4-1         -> 0.0.4-3        

Added packages:
     cookies                             2.2.1-1        
     distributed                         1.25.3-1       
     pytest-localserver                  tip-1          
     responses                           0.10.4-2       
     tblib                               1.3.2-1        
     zict                                0.1.3-1        

Removed packages:
     none

Updated bundles:
     computer-vision-basic: +PyYAML
     computer-vision-models: +PyYAML
     desktop-dev: -usbutils-dev
     desktop-kde: +kio-extras
     machine-learning-mycroft: +PyYAML
     machine-learning-tensorflow: +PyYAML
     machine-learning-web-ui: +PyYAML
     os-clr-on-clr: -usbutils-dev
     os-clr-on-clr-dev: -usbutils-dev
     os-utils-gui-dev: -usbutils-dev
     python-data-science: +PyYAML
     python-extras: +PyYAML
     sysadmin-basic-dev: -usbutils-dev
     user-basic-dev: -usbutils-dev

Added bundles:
     ocaml-basic: bash-bin ca-certs-static clr-power-tweaks
     clr-systemd-config cnf coreutils-bin dbus-autostart dbus-bin
     dist-pam-configs e2fsprogs-bin filesystem gawk-bin glibc-utils
     grep-bin hostname-bin iproute2-bin kmod-bin nano-bin nano-data
     netbase nettle-lib nss-altfiles ocaml ocamlbuild procps-ng-bin
     sed-bin shadow systemd systemd-bootchart-config systemd-lib
     tzdata-minimal util-linux-bin

Removed bundles:
     none

Changes in package R-fGarch (from 3042.83-5 to 3042.83.1-6):
     Arjan van de Ven - R-fGarch: Autospec creation for update from version 3042.83 to version 3042.83.1

Changes in package R-pamr (from 1.55-12 to 1.56-13):
     Arjan van de Ven - R-pamr: Autospec creation for update from version 1.55 to version 1.56

Changes in package R-partykit (from 1.2.2-15 to 1.2.3-16):
     Arjan van de Ven - R-partykit: Autospec creation for update from version 1.2.2 to version 1.2.3

Changes in package Shapely (from 1.6.4.post2-1 to 1.6.4.post2-2):
     Arjan van de Ven - Shapely: Autospec creation for version 1.6.4.post2

Changes in package acrn-hypervisor (from 2019w05.2.140000p-158 to 2019w05.4.140000p-159):
     Alek Du - acrn-hypervisor: Autospec creation for update from version 2019w05.2.140000p to version 2019w05.4.140000p
     Miguel Bernal Marin - remove beta patches

Changes in package atomicwrites (from 1.2.1-21 to 1.3.0-22):
     Arjan van de Ven - [update to 1.3.0] The changes in #38 broke the ability to use prefix, suffix and bufsize arguments when creating the tempfile. This commit restores the ability.

Changes in package certbot (from 0.30.2-42 to 0.30.2-44):
     Arjan van de Ven - certbot: Autospec creation for version 0.30.2
     Arjan van de Ven - certbot: Autospec creation for version 0.30.2

Changes in package clr-bundles (from 27530-34 to 27580-35):
     Arjan van de Ven - [update to 27580] Add machine-learning-pytorch bundle

Changes in package cnf (from 1-135 to 1-136):
     Arjan van de Ven - version bump from 1-135 to 1-136
     Arjan van de Ven - update command list

Changes in package colcon-core (from 0.3.15-14 to 0.3.15-15):
     Arjan van de Ven - colcon-core: Autospec creation for version 0.3.15

Changes in package dask (from 1.1.0-50 to 1.1.1-51):
     Arjan van de Ven - [update to 1.1.1] rearrange_by_column: ensure that shuffle arg defaults to 'disk' if it's None in dask.config (#4414)

Changes in package gcc (from 8.2.0-291 to 8.2.0-292):
     Arjan van de Ven - version bump from 8.2.0-291 to 8.2.0-292
     Arjan van de Ven - stable branch update

Changes in package gensim (from 3.7.0-34 to 3.7.1-35):
     Arjan van de Ven - [update to 3.7.1] Move out `process_result_queue` from cycle in `LdaMulticore` (#2358)

Changes in package glibc (from 2.28-238 to 2.29-238):
     Arjan van de Ven - update to version 2.29

Changes in package luarocks (from 3.0.4-4 to 3.0.4-5):
     Auke Kok - luarocks: Autospec creation for version 3.0.4

Changes in package optipng (from 0.7.7-1 to 0.7.7-2):
     Patrick McCarty - Fix sourceforge URL

Changes in package padatious (from 0.4.5-2 to 0.4.6-3):
     Arjan van de Ven - padatious: Autospec creation for update from version 0.4.5 to version 0.4.6

Changes in package python-editor (from 1.0.3-21 to 1.0.4-22):
     Arjan van de Ven - [update to 1.0.4] Fix suffix default (should be '', not None)

Changes in package scikit-learn (from 0.20.2-68 to 0.20.2-69):
     Arjan van de Ven - scikit-learn: Autospec creation for version 0.20.2

Changes in package spyrk (from 0.0.4-1 to 0.0.4-3):
     Arjan van de Ven - spyrk: Autospec creation for version 0.0.4
     Arjan van de Ven - spyrk: Autospec creation for version 0.0.4
     Arjan van de Ven - "update git url"
mrkz commented 5 years ago

Also, here's the difference reported by swupd verify between 27590..27600:

root@b816258ca5d9 / # swupd verify -m 27600 --fix --force
Verifying version 27600
WARNING: the force or picky option is specified; ignoring version mismatch for verify --fix
Verifying files
        ...100%
Starting download of remaining update content. This may take a while...
        ...100%
Finishing download of update content...
Adding any missing files

Fixing modified files
        ...3%
Hash mismatch for file: /usr/bin/gencat
        fixed

Hash mismatch for file: /usr/bin/getconf
        fixed

Hash mismatch for file: /usr/bin/getent
        fixed
        ...4%
Hash mismatch for file: /usr/bin/iconv
        fixed

Hash mismatch for file: /usr/bin/iconvconfig
        fixed

Hash mismatch for file: /usr/bin/ldconfig
        fixed
        ...5%
Hash mismatch for file: /usr/bin/locale
        fixed
        ...7%
Hash mismatch for file: /usr/bin/pcprofiledump
        fixed

Hash mismatch for file: /usr/bin/pldd
        fixed
        ...8%
Hash mismatch for file: /usr/bin/rpcgen
        fixed

Hash mismatch for file: /usr/bin/sotruss
        fixed

Hash mismatch for file: /usr/bin/sprof
        fixed
        ...10%
Hash mismatch for file: /usr/bin/tzselect
        fixed
        ...11%
Hash mismatch for file: /usr/bin/xtrace
        fixed

Hash mismatch for file: /usr/bin/zdump
        fixed

Hash mismatch for file: /usr/bin/zic
        fixed
        ...12%
Hash mismatch for file: /usr/lib/os-release
        fixed
        ...23%
Hash mismatch for file: /usr/lib64/audit/sotruss-lib.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ANSI_X3.110.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ARMSCII-8.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ASMO_449.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/BIG5.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/BIG5HKSCS.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/BRF.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP10007.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1125.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1250.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1251.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1252.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1253.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1254.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1255.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1256.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1257.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP1258.so
        fixed
        ...24%
Hash mismatch for file: /usr/lib64/gconv/CP737.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP770.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP771.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP772.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP773.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP774.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP775.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CP932.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CSN_369103.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/CWI.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/EUC-CN.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/EUC-JISX0213.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/EUC-JP-MS.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/EUC-JP.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/EUC-KR.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/EUC-TW.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/GEORGIAN-ACADEMY.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/GEORGIAN-PS.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/GOST_19768-74.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/GREEK-CCITT.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/GREEK7-OLD.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/GREEK7.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/HP-GREEK8.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/HP-ROMAN8.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/HP-ROMAN9.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/HP-THAI8.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/HP-TURKISH8.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/IBM858.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/IEC_P27-1.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/INIS-8.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/INIS-CYRILLIC.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/INIS.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISIRI-3342.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO-2022-CN-EXT.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO-2022-CN.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO-2022-JP-3.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO-2022-JP.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO-2022-KR.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO-IR-197.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO-IR-209.so
        fixed
        ...25%
Hash mismatch for file: /usr/lib64/gconv/ISO646.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-1.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-10.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-11.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-13.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-14.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-15.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-16.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-2.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-3.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-4.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-5.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-6.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-7.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-8.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-9.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO8859-9E.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO_10367-BOX.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO_11548-1.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO_2033.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO_5427-EXT.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO_5427.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO_5428.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO_6937-2.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/ISO_6937.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/JOHAB.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/KOI-8.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/KOI8-R.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/KOI8-RU.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/KOI8-T.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/KOI8-U.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/LATIN-GREEK-1.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/LATIN-GREEK.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/MAC-CENTRALEUROPE.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/MAC-IS.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/MAC-SAMI.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/MAC-UK.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/MACINTOSH.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/MIK.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/NATS-DANO.so
        fixed
        ...26%
Hash mismatch for file: /usr/lib64/gconv/NATS-SEFI.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/PT154.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/RK1048.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/SAMI-WS2.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/SHIFT_JISX0213.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/SJIS.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/T.61.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/TCVN5712-1.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/TIS-620.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/TSCII.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/UHC.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/UNICODE.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/UTF-16.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/UTF-32.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/UTF-7.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/VISCII.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/gconv-modules
        fixed

Hash mismatch for file: /usr/lib64/gconv/libCNS.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/libGB.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/libISOIR165.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/libJIS.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/libJISX0213.so
        fixed

Hash mismatch for file: /usr/lib64/gconv/libKSC.so
        fixed

Hash mismatch for file: /usr/lib64/glibc/getconf/POSIX_V6_LP64_OFF64
        fixed

Hash mismatch for file: /usr/lib64/glibc/getconf/POSIX_V7_LP64_OFF64
        fixed

Hash mismatch for file: /usr/lib64/glibc/getconf/XBS5_LP64_OFF64
        fixed

Hash mismatch for file: /usr/lib64/haswell/libm-2.29.so
        fixed

Hash mismatch for file: /usr/lib64/haswell/libm.so.6
        fixed
        ...27%
Hash mismatch for file: /usr/lib64/ld-2.29.so
        fixed

Hash mismatch for file: /usr/lib64/ld-linux-x86-64.so.2
        fixed

Hash mismatch for file: /usr/lib64/libBrokenLocale-2.29.so
        fixed

Hash mismatch for file: /usr/lib64/libBrokenLocale.so.1
        fixed

Hash mismatch for file: /usr/lib64/libSegFault.so
        fixed

Hash mismatch for file: /usr/lib64/libanl-2.29.so
        fixed

Hash mismatch for file: /usr/lib64/libanl.so.1
        fixed

Hash mismatch for file: /usr/lib64/libc-2.29.so
        fixed

Hash mismatch for file: /usr/lib64/libc.so.6
        fixed
        ...28%
Hash mismatch for file: /usr/lib64/libcrypt-2.29.so
        fixed

Hash mismatch for file: /usr/lib64/libcrypt.so.1
        fixed

Hash mismatch for file: /usr/lib64/libdl-2.29.so
        fixed

Hash mismatch for file: /usr/lib64/libdl.so.2
        fixed
        ...30%
Hash mismatch for file: /usr/lib64/libm-2.29.so
        fixed

Hash mismatch for file: /usr/lib64/libm.so.6
        fixed

Hash mismatch for file: /usr/lib64/libmemusage.so
        fixed

Hash mismatch for file: /usr/lib64/libmvec-2.29.so
        fixed

Hash mismatch for file: /usr/lib64/libmvec.so.1
        fixed

Hash mismatch for file: /usr/lib64/libnsl-2.29.so
        fixed

Hash mismatch for file: /usr/lib64/libnsl.so.1
        fixed

Hash mismatch for file: /usr/lib64/libnss_compat-2.29.so
        fixed
        ...31%
Hash mismatch for file: /usr/lib64/libnss_compat.so.2
        fixed

Hash mismatch for file: /usr/lib64/libnss_dns-2.29.so
        fixed

Hash mismatch for file: /usr/lib64/libnss_dns.so.2
        fixed

Hash mismatch for file: /usr/lib64/libnss_files-2.29.so
        fixed

Hash mismatch for file: /usr/lib64/libnss_files.so.2
        fixed

Hash mismatch for file: /usr/lib64/libnss_hesiod-2.29.so
        fixed

Hash mismatch for file: /usr/lib64/libnss_hesiod.so.2
        fixed

Hash mismatch for file: /usr/lib64/libpcprofile.so
        fixed

Hash mismatch for file: /usr/lib64/libpthread-2.29.so
        fixed
        ...32%
Hash mismatch for file: /usr/lib64/libpthread.so.0
        fixed

Hash mismatch for file: /usr/lib64/libresolv-2.29.so
        fixed

Hash mismatch for file: /usr/lib64/libresolv.so.2
        fixed

Hash mismatch for file: /usr/lib64/librt-2.29.so
        fixed

Hash mismatch for file: /usr/lib64/librt.so.1
        fixed

Hash mismatch for file: /usr/lib64/libthread_db-1.0.so
        fixed

Hash mismatch for file: /usr/lib64/libutil-2.29.so
        fixed

Hash mismatch for file: /usr/lib64/libutil.so.1
        fixed
        ...51%
Hash mismatch for file: /usr/share/clear/commandlist.csv
        fixed

Hash mismatch for file: /usr/share/clear/version
        fixed

Hash mismatch for file: /usr/share/clear/versionstamp
        fixed

Hash mismatch for file: /usr/share/clr-bundles/cloud-api
        fixed

Hash mismatch for file: /usr/share/clr-bundles/cloud-native-basic
        fixed
        ...52%
Hash mismatch for file: /usr/share/clr-bundles/desktop-assets
        fixed

Hash mismatch for file: /usr/share/clr-bundles/desktop-kde
        fixed

Hash mismatch for file: /usr/share/clr-bundles/dev-utils-gui
        fixed
        ...54%
Hash mismatch for file: /usr/share/clr-bundles/games
        fixed
        ...55%
Hash mismatch for file: /usr/share/clr-bundles/machine-learning-basic
        fixed

Hash mismatch for file: /usr/share/clr-bundles/maker-3dprinting
        fixed

Hash mismatch for file: /usr/share/clr-bundles/network-basic-dev
        fixed
        ...56%
Hash mismatch for file: /usr/share/clr-bundles/os-clr-on-clr-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/os-utils-gui
        fixed

Hash mismatch for file: /usr/share/clr-bundles/os-utils-gui-dev
        fixed

Hash mismatch for file: /usr/share/clr-bundles/packages
        fixed

Hash mismatch for file: /usr/share/clr-bundles/perl-basic
        fixed

Hash mismatch for file: /usr/share/clr-bundles/perl-basic-dev
        fixed
        ...57%
Hash mismatch for file: /usr/share/clr-bundles/qt-basic
        fixed

Hash mismatch for file: /usr/share/clr-bundles/qt-basic-dev
        fixed
        ...58%
Hash mismatch for file: /usr/share/clr-bundles/xfce4-desktop
        fixed
        ...63%
Hash mismatch for file: /usr/share/locale/en_US.UTF-8/LC_TIME
        fixed
        ...100%
File that should be deleted: /usr/lib64/libutil-2.28.so
        deleted
File that should be deleted: /usr/lib64/librt-2.28.so
        deleted
File that should be deleted: /usr/lib64/libresolv-2.28.so
        deleted
File that should be deleted: /usr/lib64/libpthread-2.28.so
        deleted
File that should be deleted: /usr/lib64/libnss_hesiod-2.28.so
        deleted
File that should be deleted: /usr/lib64/libnss_files-2.28.so
        deleted
File that should be deleted: /usr/lib64/libnss_dns-2.28.so
        deleted
File that should be deleted: /usr/lib64/libnss_compat-2.28.so
        deleted
File that should be deleted: /usr/lib64/libnsl-2.28.so
        deleted
File that should be deleted: /usr/lib64/libmvec-2.28.so
        deleted
File that should be deleted: /usr/lib64/libm-2.28.so
        deleted
File that should be deleted: /usr/lib64/libdl-2.28.so
        deleted
File that should be deleted: /usr/lib64/libcrypt-2.28.so
        deleted
File that should be deleted: /usr/lib64/libc-2.28.so
        deleted
File that should be deleted: /usr/lib64/libanl-2.28.so
        deleted
File that should be deleted: /usr/lib64/libBrokenLocale-2.28.so
        deleted
File that should be deleted: /usr/lib64/ld-2.28.so
        deleted
File that should be deleted: /usr/lib64/haswell/libm-2.28.so
        deleted
Inspected 3986 files
  203 files did not match
    203 of 203 files were fixed
    0 of 203 files were not fixed
  18 files found which should be deleted
    18 of 18 files were deleted
    0 of 18 files were not deleted
Calling post-update helper scripts.
none
[ERROR] cbm (../src/lib/files.c:L165): Invalid block device: /
[ERROR] cbm (../src/lib/system_stub.c:L31): Invalid block device: 0:52
Out of memoryFix successful
lsandov1 commented 5 years ago

@mrkz thanks for you help. Basically I get to the same conclusion (https://github.com/lsandoval/docker-swupd-update/blob/master/README.md): 27600 version introduce the issue

mythi commented 5 years ago

This seems to happen only when the host is Clear linux too (i.e., with Clear's docker)

clr@clr-6625b1847e824b9190f48a50b3be183e ~ $ docker run -it clearlinux
root@57e52f083919 / # swupd update
Warning: Unable to verify server SSL certificate
Trying fallback CA path /usr/share/ca-certs/.prebuilt-store/anchors
Update started.
Preparing to update from 27670 to 28260
Downloading packs...

Extracting libglib pack for version 28220
        ...28%
Extracting libstdcpp pack for version 28260
        ...100%

Statistics for going from version 27670 to version 28260:

    changed bundles   : 3
    new bundles       : 4
    deleted bundles   : 0

    changed files     : 1130
    new files         : 2994
    deleted files     : 415

Starting download of remaining update content. This may take a while...
        ...100%
Finishing download of update content...
Staging file content
Applying update
        ...100%
Update was applied.
Calling post-update helper scripts.
none
[ERROR] cbm (../src/lib/files.c:L165): Invalid block device: /
[ERROR] cbm (../src/lib/system_stub.c:L31): Invalid block device: 0:39
Out of memoryUpdate took 40.5 seconds
1646 files were not in a pack
Update successful. System updated from version 27670 to version 28260
root@57e52f083919 / # exit
exit
clr@clr-6625b1847e824b9190f48a50b3be183e ~ $ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
<none>              <none>              94929131751a        47 seconds ago      230MB
clearlinux          latest              ceea98458f88        4 weeks ago         109MB
clr@clr-6625b1847e824b9190f48a50b3be183e ~ $ docker run -it 94929131751a /bin/bash
root@876f904aa2d0 / # swupd check-update
swupd: error while loading shared libraries: libsystemd.so.0: cannot open shared object file: No such file or directory
root@876f904aa2d0 / #
phmccarty commented 5 years ago

This issue looks suspiciously similar to the issue I discovered in docker a long time ago that only occurred when using the overlay storage driver instead of overlay2. See https://github.com/moby/moby/issues/31805 for background.

I will try to reproduce.

iphutch commented 5 years ago

@phmccarty please close if you cannot reproduce

DnPlas commented 5 years ago

@bryteise @phmccarty @mythi I also see something similar to this error message:

root@876f904aa2d0 / # swupd check-update swupd: error while loading shared libraries: libsystemd.so.0: cannot open shared object file: No such file or directory root@876f904aa2d0 / #

But in my case is when I run python, then try to import cv2:

#   File "/opt/conda/lib/python3.7/site-packages/cv2/__init__.py", line 3, in <module>
#     from .cv2 import *
# ImportError: libgthread-2.0.so.0: cannot open shared object file: No such file or directory

Clearlinux version: 28970

phmccarty commented 5 years ago

@DnPlas Can you check to see when /usr/lib64/libgthread-2.0.so.0 was deleted in your image? It's easiest to debug these issues when we know the point in time the file existed, and search forward in history from there.

DnPlas commented 5 years ago

@phmccarty The weird part is that it is still there:

root@bb23c3721c34 /workspace # ls /usr/lib64/ | grep libgthread
libgthread-2.0.so.0
libgthread-2.0.so.0.6000.1

Actually, the error message stops after running ldconfig. Do you think I should open a new bug for this?

Daniela

mrkz commented 5 years ago

Actually, the error message stops after running ldconfig. Do you think I should open a new bug for this?

@DnPlas please do

miguelinux commented 5 years ago

Actually, the error message stops after running ldconfig. Do you think I should open a new bug for this?

@DnPlas please do

This is a kind of a known issue with swupd+update+containers.

But yes, open a new bug issue please.

phmccarty commented 5 years ago

I have confirmed that this issue is the same as the ldconfig issue, which was addressed recently (build 29110), as swupd will now run ldconfig itself prior to running update triggers.

Once we update the Clear Linux OS base Docker image, we can close this issue.

mrkz commented 5 years ago

as @phmccarty mentions, once https://github.com/docker-library/official-images/pull/5873 is merged, this should not be reproducible anymore and we should be able to close this issue.

phmccarty commented 5 years ago

The above referenced PR is merged. Closing.

mrkz commented 5 years ago

Just for reference, I cannot reproduce buggy behavior as mentioned in this issue description:

% docker pull clearlinux
Using default tag: latest
latest: Pulling from library/clearlinux
81cf4cf60965: Pull complete 
Digest: sha256:56a90c9433d3c27e588804b34dd37035f108a24c963be61dcc165bf5b21c6406
Status: Downloaded newer image for clearlinux:latest
 % docker run -it clearlinux
root@62585f86d80e / # cat /usr/share/clear/version
29250
root@fc60a90364b4 / # swupd update
Update started.
Preparing to update from 29250 to 29280
Downloading packs...
    ...100%

Extracting os-core-update pack for version 29270

Statistics for going from version 29250 to version 29280:

    changed bundles   : 3
    new bundles       : 0
    deleted bundles   : 0

    changed files     : 133
    new files         : 2
    deleted files     : 4

Starting download of remaining update content. This may take a while...
    ...100%
Finishing download of update content...
Staging file content
Applying update
    ...100%
Update was applied.
Calling post-update helper scripts.
none
[ERROR] cbm (../src/lib/files.c:L165): Invalid block device: /
[ERROR] cbm (../src/lib/system_stub.c:L31): Invalid block device: 0:47
Out of memoryUpdate took 10.8 seconds, 8 MB transferred
123 files were not in a pack
Update successful. System updated from version 29250 to version 29280
root@fc60a90364b4 / #

Later on, in other prompt:

% docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED              STATUS              PORTS               NAMES
fc60a90364b4        clearlinux          "/bin/bash"         About a minute ago   Up About a minute                       reverent_fermi
% docker commit fc60a90364b4
% docker run -it dc7e106a11edb03849b418d40d410d7b341e28052bb42f4e3455506dd3c44c9e /bin/bash
root@7cc2157a42f8 / #