search

project-copacetic / copacetic

🧵 CLI tool for directly patching container images!
https://project-copacetic.github.io/copacetic/
Apache License 2.0
1.03k stars 69 forks source link

[BUG] copa CLI gets to a deadlock if package validation fails #503

Closed mhrabovcin closed 8 months ago

mhrabovcin commented 9 months ago

Version of copa

v0.6.0

Expected Behavior

CLI should propagate error and exist with appropriate status code.

Actual Behavior

CLI hangs until the provided patch timeout is reached.

https://github.com/project-copacetic/copacetic/blob/2f621d177d17fa6ee73dffc2d3aea58ac89688d2/pkg/patch/patch.go#L46-L61

Steps To Reproduce

Trivy scan:

trivy image cr.fluentbit.io/fluent/fluent-bit:2.1.4 --vuln-type os --ignore-unfixed -f json -o fluent-bit.2.1.4.json

Patch:

copa patch -i cr.fluentbit.io/fluent/fluent-bit:2.1.4 -r fluent-bit.2.1.4.json -t 2.1.4-patched

Output:

time="2024-02-14T10:56:44Z" level=debug msg="updates to apply: &{{{debian 11.7} {amd64}} [{libc6 2.31-13+deb11u6 2.31-13+deb11u7 CVE-2023-4911} {libgnutls30 3.7.1-5+deb11u3 3.7.1-5+deb11u5 CVE-2024-0567} {libgnutls30 3.7.1-5+deb11u3 3.7.1-5+deb11u5 CVE-2023-5981} {libgssapi-krb5-2 1.18.3-6+deb11u3 1.18.3-6+deb11u4 CVE-2023-36054} {libk5crypto3 1.18.3-6+deb11u3 1.18.3-6+deb11u4 CVE-2023-36054} {libkrb5-3 1.18.3-6+deb11u3 1.18.3-6+deb11u4 CVE-2023-36054} {libkrb5support0 1.18.3-6+deb11u3 1.18.3-6+deb11u4 CVE-2023-36054} {libpq5 13.11-0+deb11u1 13.13-0+deb11u1 CVE-2023-39417} {libpq5 13.11-0+deb11u1 13.13-0+deb11u1 CVE-2023-5869} {libpq5 13.11-0+deb11u1 13.13-0+deb11u1 CVE-2023-5868} {libpq5 13.11-0+deb11u1 13.13-0+deb11u1 CVE-2023-5870} {libssl1.1 1.1.1n-0+deb11u4 1.1.1n-0+deb11u5 CVE-2023-0464} {libssl1.1 1.1.1n-0+deb11u4 1.1.1n-0+deb11u5 CVE-2023-0465} {libssl1.1 1.1.1n-0+deb11u4 1.1.1n-0+deb11u5 CVE-2023-0466} {libssl1.1 1.1.1n-0+deb11u4 1.1.1n-0+deb11u5 CVE-2023-2650} {libssl1.1 1.1.1n-0+deb11u4 1.1.1v-0~deb11u1 CVE-2023-3446} {libssl1.1 1.1.1n-0+deb11u4 1.1.1v-0~deb11u1 CVE-2023-3817} {openssl 1.1.1n-0+deb11u4 1.1.1n-0+deb11u5 CVE-2023-0464} {openssl 1.1.1n-0+deb11u4 1.1.1n-0+deb11u5 CVE-2023-0465} {openssl 1.1.1n-0+deb11u4 1.1.1n-0+deb11u5 CVE-2023-0466} {openssl 1.1.1n-0+deb11u4 1.1.1n-0+deb11u5 CVE-2023-2650} {openssl 1.1.1n-0+deb11u4 1.1.1v-0~deb11u1 CVE-2023-3446} {openssl 1.1.1n-0+deb11u4 1.1.1v-0~deb11u1 CVE-2023-3817}]}"
time="2024-02-14T10:56:44Z" level=debug msg="Trying docker driver"
time="2024-02-14T10:56:44Z" level=debug msg="serving grpc connection"
time="2024-02-14T10:56:44Z" level=debug msg="stopping session"
time="2024-02-14T10:56:44Z" level=debug msg="Could not use docker driver" error="failed to solve: rpc error: code = Unknown desc = requested experimental feature mergeop  is not supported by build server, please update \nrequested experimental feature diffop  is not supported by build server, please update \nmissing required buildkit functionality"
time="2024-02-14T10:56:44Z" level=debug msg="Trying buildx driver"
time="2024-02-14T10:56:48Z" level=debug msg="Connect to buildx instance" driver=docker-container endpoint="unix:///var/run/docker.sock" name=copacetic-action0
time="2024-02-14T10:56:48Z" level=debug msg="serving grpc connection"
time="2024-02-14T10:56:48Z" level=debug msg="stopping session"
time="2024-02-14T10:56:48Z" level=debug msg="serving grpc connection"
#1 resolve image config for cr.fluentbit.io/fluent/fluent-bit:2.1.4
#1 DONE 0.4s
time="2024-02-14T10:56:49Z" level=debug msg="Using debian:11-slim as basis for tooling image"

#1 resolve image config for cr.fluentbit.io/fluent/fluent-bit:2.1.4
#1 DONE 1.0s

#2 docker-image://cr.fluentbit.io/fluent/fluent-bit:2.1.4
#2 resolve cr.fluentbit.io/fluent/fluent-bit:2.1.4 0.1s done
#2 DONE 0.4s

#3 docker-image://docker.io/library/debian:11-slim
#3 resolve docker.io/library/debian:11-slim
#3 resolve docker.io/library/debian:11-slim 1.1s done
#3 sha256:5d0aeceef7eeb53c3f853fb229ea7fd13a5a56f4ba371ca48f0477493046b702 5.24MB / 31.42MB 0.2s
#3 sha256:5d0aeceef7eeb53c3f853fb229ea7fd13a5a56f4ba371ca48f0477493046b702 31.42MB / 31.42MB 0.4s done
#3 ...

#2 docker-image://cr.fluentbit.io/fluent/fluent-bit:2.1.4
#2 sha256:51bb3c3cd8081d66f32e6102e9092512245826fe75af6ee705dc727e02c4dbe9 14.56MB / 14.56MB 0.5s done
#2 sha256:852c2006aee6e783c44098bd02554ed55be750c534680e4dd2e2a6bea0e58293 5.82MB / 5.82MB 0.5s done
#2 sha256:a5bce306db56202cff4b1f22b5a3347c56ff1f377a8391f7a3039b8f51a82f3f 123.70kB / 123.70kB 0.4s done
#2 sha256:4eec690774a46467a912715848c71dbbdb049008b2252432155522a7f9ccfa92 52.43kB / 52.43kB 0.8s done
#2 sha256:52907d314ddce378f3f36e26629baef60c71d72a0620b9d31c47c8cb9de6467e 741.34kB / 741.34kB 0.1s done
#2 sha256:dfc02eb7708f919bb3b56c008561e4430ea87cd33bc93cb65c2c3c7f0908e5cf 131.16kB / 131.16kB 0.1s done
#2 sha256:f51c8fa85103a5c3ac5ec1969b693ccc0dc2b409dec106e4025e31c436c0c740 961.97kB / 961.97kB 0.1s done
#2 sha256:96266735468f361ae6828901a80fc15a7f75e26640351df9e0f0f9824f36cf92 5.85MB / 5.85MB 0.1s done
#2 sha256:80ec95682aee84c463cbf19b82eef5e1707c5e31b7be9750e1a90bff8b4183ee 2.06MB / 2.06MB 0.1s done
#2 sha256:5627a970d25e752d971a501ec7e35d0d6fdcd4a3ce9e958715a686853024794a 130.56kB / 130.56kB 0.1s done
#2 sha256:7c881f9ab25e0d86562a123b5fb56aebf8aa0ddd7d48ef602faf8d1e7cf43d8c 355B / 355B 0.1s done
#2 sha256:4aa0ea1413d37a58615488592a0b827ea4b2e48fa5a77cf707d0e35f025e613f 385B / 385B 0.1s done
#2 sha256:1e3d9b7d145208fa8fa3ee1c9612d0adaac7255f1bbc9ddea7e461e0b317805c 113B / 113B 0.1s done
#2 sha256:e8c73c638ae9ec5ad70c49df7e484040d889cca6b4a9af056579c3d058ea93f0 198B / 198B 0.1s done
#2 sha256:fcb6f6d2c9986d9cd6a2ea3cc2936e5fc613e09f1af9042329011e43057f3265 317B / 317B 0.1s done
#2 sha256:fe5ca62666f04366c8e7f605aa82997d71320183e99962fa76b3209fdfbb8b58 21.20kB / 21.20kB 0.1s done
#2 sha256:a7ca0d9ba68fdce7e15bc0952d3e898e970548ca24d57698725836c039086639 103.73kB / 103.73kB 0.1s done
#2 extracting sha256:a7ca0d9ba68fdce7e15bc0952d3e898e970548ca24d57698725836c039086639 0.0s done
#2 sha256:b02a7525f878e61fc1ef8a7405a2cc17f866e8de222c1c98fd6681aff6e509db 716.49kB / 716.49kB 0.2s done
#2 extracting sha256:fe5ca62666f04366c8e7f605aa82997d71320183e99962fa76b3209fdfbb8b58 0.0s done
#2 extracting sha256:b02a7525f878e61fc1ef8a7405a2cc17f866e8de222c1c98fd6681aff6e509db 0.2s done
#2 extracting sha256:fcb6f6d2c9986d9cd6a2ea3cc2936e5fc613e09f1af9042329011e43057f3265 0.0s done
#2 DONE 1.6s

#3 docker-image://docker.io/library/debian:11-slim
#3 extracting sha256:5d0aeceef7eeb53c3f853fb229ea7fd13a5a56f4ba371ca48f0477493046b702
#3 extracting sha256:5d0aeceef7eeb53c3f853fb229ea7fd13a5a56f4ba371ca48f0477493046b702 0.8s done
#3 DONE 2.3s

#2 docker-image://cr.fluentbit.io/fluent/fluent-bit:2.1.4
#2 extracting sha256:e8c73c638ae9ec5ad70c49df7e484040d889cca6b4a9af056579c3d058ea93f0 0.0s done
#2 extracting sha256:1e3d9b7d145208fa8fa3ee1c9612d0adaac7255f1bbc9ddea7e461e0b317805c 0.0s done
#2 extracting sha256:4aa0ea1413d37a58615488592a0b827ea4b2e48fa5a77cf707d0e35f025e613f 0.0s done
#2 extracting sha256:7c881f9ab25e0d86562a123b5fb56aebf8aa0ddd7d48ef602faf8d1e7cf43d8c 0.0s done
#2 extracting sha256:5627a970d25e752d971a501ec7e35d0d6fdcd4a3ce9e958715a686853024794a 0.0s done
#2 extracting sha256:96266735468f361ae6828901a80fc15a7f75e26640351df9e0f0f9824f36cf92 0.1s done
#2 extracting sha256:80ec95682aee84c463cbf19b82eef5e1707c5e31b7be9750e1a90bff8b4183ee 0.0s done
#2 extracting sha256:f51c8fa85103a5c3ac5ec1969b693ccc0dc2b409dec106e4025e31c436c0c740 0.0s done
#2 extracting sha256:dfc02eb7708f919bb3b56c008561e4430ea87cd33bc93cb65c2c3c7f0908e5cf 0.0s done
#2 extracting sha256:52907d314ddce378f3f36e26629baef60c71d72a0620b9d31c47c8cb9de6467e 0.0s done
#2 extracting sha256:4eec690774a46467a912715848c71dbbdb049008b2252432155522a7f9ccfa92 0.0s done
#2 extracting sha256:852c2006aee6e783c44098bd02554ed55be750c534680e4dd2e2a6bea0e58293 0.2s done
#2 extracting sha256:a5bce306db56202cff4b1f22b5a3347c56ff1f377a8391f7a3039b8f51a82f3f 0.0s done
#2 extracting sha256:51bb3c3cd8081d66f32e6102e9092512245826fe75af6ee705dc727e02c4dbe9
#2 extracting sha256:51bb3c3cd8081d66f32e6102e9092512245826fe75af6ee705dc727e02c4dbe9 0.2s done
#2 DONE 2.4s

#4 apt update
#4 0.062 
#4 0.062 WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
#4 0.062 
#4 0.103 Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
#4 0.131 Get:2 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
#4 0.142 Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
#4 0.202 Get:4 http://deb.debian.org/debian bullseye/main amd64 Packages [8068 kB]
#4 0.298 Get:5 http://deb.debian.org/debian-security bullseye-security/main amd64 Packages [268 kB]
#4 0.323 Get:6 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [18.8 kB]
#4 1.213 Fetched 8562 kB in 1s (7506 kB/s)
#4 1.213 Reading package lists...
#4 1.589 Building dependency tree...
#4 1.700 Reading state information...
#4 1.706 All packages are up to date.
#4 DONE 2.3s

#5 apt install busybox-static
#5 0.080 
#5 0.080 WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
#5 0.080 
#5 0.085 Reading package lists...
#5 0.468 Building dependency tree...
#5 0.561 Reading state information...
#5 0.658 The following NEW packages will be installed:
#5 0.658   busybox-static
#5 0.690 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
#5 0.690 Need to get 920 kB of archives.
#5 0.690 After this operation, 2111 kB of additional disk space will be used.
#5 0.690 Get:1 http://deb.debian.org/debian bullseye/main amd64 busybox-static amd64 1:1.30.1-6+b3 [920 kB]
#5 0.809 debconf: delaying package configuration, since apt-utils is not installed
#5 0.827 Fetched 920 kB in 0s (14.5 MB/s)
#5 0.843 Selecting previously unselected package busybox-static.
#5 0.843 (Reading database ... 
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 6670 files and directories currently installed.)
#5 0.847 Preparing to unpack .../busybox-static_1%3a1.30.1-6+b3_amd64.deb ...
#5 0.851 Unpacking busybox-static (1:1.30.1-6+b3) ...
#5 0.943 Setting up busybox-static (1:1.30.1-6+b3) ...
#5 DONE 1.0s

#6 copy /bin/busybox /bin/busybox
#6 DONE 0.0s

#7 mkdir /copa-out
#7 DONE 0.0s

#8 /bin/busybox sh -c 
                status="$DPKG_STATUS_IS_UNKNOWN"
                if [ -f "$DPKG_STATUS_PATH" ]; then
                    status="$DPKG_STATUS_IS_FILE"
                    cp "$DPKG_STATUS_PATH" "$RESULTS_PATH"
                elif [ -d "$DPKG_STATUS_FOLDER" ]; then
                    status="$DPKG_STATUS_IS_DIRECTORY"
                    ls -1 "$DPKG_STATUS_FOLDER" > "$RESULT_STATUSD_PATH"
                fi
                echo -n "$status" > "${RESULTS_PATH}/${STATUSD_OUTPUT_FILENAME}"

#8 DONE 0.1s
time="2024-02-14T10:56:55Z" level=info msg="Processed status.d: base-files base-files.md5sums ca-certificates libatomic1 libc6 libc6.md5sums libcom-err2 libffi7 libgcc-s1 libgcc-s1.md5sums libgcrypt20 libgmp10 libgnutls30 libgomp1 libgomp1.md5sums libgpg-error0 libgssapi-krb5-2 libhogweed6 libidn2-0 libk5crypto3 libkeyutils1 libkrb5-3 libkrb5support0 libldap-2.4-2 liblz4-1 liblzma5 libnettle8 libp11-kit0 libpq5 libsasl2-2 libssl1.1 libssl1.1.md5sums libstdc++6 libstdc++6.md5sums libsystemd0 libtasn1-6 libunistring2 libyaml-0-2 libzstd1 netbase netbase.md5sums openssl openssl.md5sums pkg-config tzdata tzdata.md5sums zlib1g"

#9 apt download --no-install-recommends libkrb5support0 libpq5 libc6 libgnutls30 libgssapi-krb5-2 libk5crypto3 libkrb5-3 libssl1.1 openssl
#9 0.078 
#9 0.078 WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
#9 0.078 
#9 0.647 Get:1 http://deb.debian.org/debian bullseye/main amd64 libc6 amd64 2.31-13+deb11u8 [2823 kB]
#9 0.708 Get:2 http://deb.debian.org/debian bullseye/main amd64 libgnutls30 amd64 3.7.1-5+deb11u4 [1340 kB]
#9 0.714 Get:3 http://deb.debian.org/debian bullseye/main amd64 libgssapi-krb5-2 amd64 1.18.3-6+deb11u4 [165 kB]
#9 0.715 Get:4 http://deb.debian.org/debian bullseye/main amd64 libk5crypto3 amd64 1.18.3-6+deb11u4 [114 kB]
#9 0.716 Get:5 http://deb.debian.org/debian bullseye/main amd64 libkrb5-3 amd64 1.18.3-6+deb11u4 [363 kB]
#9 0.718 Get:6 http://deb.debian.org/debian bullseye/main amd64 libkrb5support0 amd64 1.18.3-6+deb11u4 [65.6 kB]
#9 0.719 Get:7 http://deb.debian.org/debian bullseye/main amd64 libssl1.1 amd64 1.1.1w-0+deb11u1 [1566 kB]
#9 0.726 Get:8 http://deb.debian.org/debian bullseye/main amd64 openssl amd64 1.1.1w-0+deb11u1 [859 kB]
#9 0.730 Get:9 http://deb.debian.org/debian bullseye/main amd64 libpq5 amd64 13.13-0+deb11u1 [182 kB]
#9 0.735 Fetched 7477 kB in 0s (65.5 MB/s)
#9 0.735 W: Download is performed unsandboxed as root as file '/copa-downloads/libc6_2.31-13+deb11u8_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
#9 DONE 0.8s

#10 mkdir /copa-out
#10 DONE 0.0s

#11 mkdir /var/lib/dpkg/status.d
#11 DONE 0.0s

#12 find . -name *.deb -exec sh -c dpkg-deb -f {} > /copa-out/{}.fields ;
#12 DONE 0.1s

#13 find . -name *.fields -exec sh -c grep "^Package:\|^Version:" {} >> results.manifest ;
#13 DONE 0.1s

#14 diff (find . -name *.deb -exec sh -c dpkg-deb -f {} > /copa-out/{}.fields ;) -> (find . -name *.fields -exec sh -c grep "^Package:\|^Version:" {} >> results.manifest ;)
#14 DONE 0.0s
time="2024-02-14T10:56:56Z" level=info msg="Validated package libkrb5support0 version 1.18.3-6+deb11u4 meets requested version 1.18.3-6+deb11u4"
time="2024-02-14T10:56:56Z" level=info msg="Validated package libpq5 version 13.13-0+deb11u1 meets requested version 13.13-0+deb11u1"
time="2024-02-14T10:56:56Z" level=info msg="Validated package libc6 version 2.31-13+deb11u8 meets requested version 2.31-13+deb11u7"
time="2024-02-14T10:56:56Z" level=error msg="downloaded package libgnutls30 version 3.7.1-5+deb11u4 lower than required 3.7.1-5+deb11u5 for update"
time="2024-02-14T10:56:56Z" level=info msg="Validated package libgssapi-krb5-2 version 1.18.3-6+deb11u4 meets requested version 1.18.3-6+deb11u4"
time="2024-02-14T10:56:56Z" level=info msg="Validated package libk5crypto3 version 1.18.3-6+deb11u4 meets requested version 1.18.3-6+deb11u4"
time="2024-02-14T10:56:56Z" level=info msg="Validated package libkrb5-3 version 1.18.3-6+deb11u4 meets requested version 1.18.3-6+deb11u4"
time="2024-02-14T10:56:56Z" level=info msg="Validated package libssl1.1 version 1.1.1w-0+deb11u1 meets requested version 1.1.1v-0~deb11u1"
time="2024-02-14T10:56:56Z" level=info msg="Validated package openssl version 1.1.1w-0+deb11u1 meets requested version 1.1.1v-0~deb11u1"
time="2024-02-14T10:56:56Z" level=debug msg="stopping session"

#14 diff (find . -name *.deb -exec sh -c dpkg-deb -f {} > /copa-out/{}.fields ;) -> (find . -name *.fields -exec sh -c grep "^Package:\|^Version:" {} >> results.manifest ;)
#14 diffing 0.0s done
#14 DONE 0.0s
time="2024-02-14T11:01:45Z" level=error msg="patch exceeded timeout 5m0s"
Error: patch exceeded timeout 5m0s

The build completes if I provide --ignore-errors flag.

Are you willing to submit PRs to contribute to this bug fix?

sozercan commented 9 months ago

Thanks for the report! This seems like a debian db bug since there's no libgnutls30 3.7.1-5+deb11u5 https://salsa.debian.org/security-tracker-team/security-tracker/-/blob/master/data/CVE/list?ref_type=heads#L5744 https://packages.debian.org/bullseye/libgnutls30 latest is 3.7.1-5+deb11u4 as of now Mitigation is to set up a policy to ignore CVE-2024-0567 or use --ignore-errors for now

This should not lock copa though. That is a regression from v0.5.0 (guessing from #381) @ashnamehrotra can you ptal at this?

mhrabovcin commented 9 months ago

Thanks for providing the workaround. I can confirm that --ignore-errors flag works for given image.