search

project-copacetic / copacetic

🧵 CLI tool for directly patching container images!
https://project-copacetic.github.io/copacetic/
Apache License 2.0
933 stars 62 forks source link

[BUG] ERROR exporting to docker image format #339

Closed lioryantov closed 8 months ago

lioryantov commented 11 months ago

Version of copa

copa version 0.4.1

Expected Behavior

Expected copa to patch the image based on trivy scan results.

Actual Behavior

Getting error during patch process: ERROR exporting to docker image format

Steps To Reproduce


$ trivy version
Version: 0.45.1
Vulnerability DB:
  Version: 2
  UpdatedAt: 2023-09-30 06:20:06.92472101 +0000 UTC
  NextUpdate: 2023-09-30 12:20:06.92472051 +0000 UTC
  DownloadedAt: 2023-09-30 11:01:25.072766517 +0000 UTC
$ copa --version
copa version 0.4.1

Run moby/buildkit:v0.12.2 docker: 

$ docker run \
  --detach \
  --rm \
  --privileged \
  -p 127.0.0.1:8888:8888/tcp \
  --name buildkitd \
  --entrypoint buildkitd \
  "moby/buildkit:v0.12.2" \
  --addr tcp://0.0.0.0:8888

I replaced in output scanned image with "my-image":

Run trivy to get scan results:

$ trivy image --vuln-type os --ignore-unfixed -f json -o copa-patch.json $my-image
2023-09-30T14:01:15.359+0300    INFO    Need to update DB
2023-09-30T14:01:15.359+0300    INFO    DB Repository: ghcr.io/aquasecurity/trivy-db
2023-09-30T14:01:15.359+0300    INFO    Downloading DB...
40.10 MiB / 40.10 MiB [-------------------------------------------------------------------------------------------------------------------------] 100.00% 5.22 MiB p/s 7.9s
2023-09-30T14:01:25.073+0300    INFO    Vulnerability scanning is enabled
2023-09-30T14:01:25.073+0300    INFO    Secret scanning is enabled
2023-09-30T14:01:25.073+0300    INFO    If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2023-09-30T14:01:25.073+0300    INFO    Please see also https://aquasecurity.github.io/trivy/v0.45/docs/scanner/secret/#recommendation for faster secret detection
2023-09-30T14:01:35.687+0300    INFO    Detected OS: alpine
2023-09-30T14:01:35.687+0300    INFO    Detecting Alpine vulnerabilities...

Running copa patch command: 

$ copa patch \
     -i $my-image \
     -r copa-patch.json \
     -t $my-image-patched \
     -a tcp://0.0.0.0:8888

[+] Building 33.8s (8/8) FINISHED                                                                                                                                          
 => docker-image://my-image                                                                                              20.8s
 => => resolve my-image                                                                                                   2.0s
 => => sha256:c9d7ead20553853875b36959215175c9a89c23ce08f0e17da02600a1b0e8a5c6 183.08MB / 183.08MB                                                                   13.3s
 => => sha256:170e4e13009e9b1d243e32f065068f72c223fbeafc045495da6c010b32a2ab95 1.57MB / 1.57MB                                                                        2.3s
 => => sha256:00b38c0d6d7e62793aa6c4200f3c5e404949c34f8a18629ab5ecdec2cdf6a349 127.71kB / 127.71kB                                                                    1.8s
 => => sha256:fe25c66a9a3122454dc7f33ecfc88233fc08ad6b49f9f35c0e1b476adc60f617 82.32kB / 82.32kB                                                                      1.6s
 => => sha256:30e3f748b11ffca52e071d6ca397a39cc9ee191436a5e7fb8ea547c677cb7e41 146.33MB / 146.33MB                                                                   10.4s
 => => sha256:539d3b93165bbb621c904215f176b90cf5854928fa446618f8af28821e78127a 214B / 214B                                                                            0.4s
 => => sha256:3e34833d144bccde30401d665ae65ba571447a39a04c2cc02b10a81e8858ce9d 356B / 356B                                                                            0.4s
 => => sha256:86787f6d1cfdad7606d38054a7b28cad6ceabf6c13011c2876c1a7dff5278917 34.96MB / 34.96MB                                                                      2.9s
 => => sha256:994e83f716a0db024fb37caf707eae5af27172a0fffb691c6e7b53bb7fc5b3ab 9.28MB / 9.28MB                                                                        2.2s
 => => sha256:c2fb05fc177834cb351a4f6f3d4e204f9497b99bd9d1f812e7067bc185d9e82d 1.33kB / 1.33kB                                                                        0.5s
 => => sha256:041526fa6c45ea6a1845a0cbdc7edd6eefbe586959246af11fc0200d562b5816 57.91MB / 57.91MB                                                                      4.1s
 => => sha256:5cf5f0762cf7c3135a3c63cc4711ca56eef959f0cfe5731febc20b271e09ff12 129B / 129B                                                                            0.5s
 => => sha256:6a2bf8aa57bd3ac37dc6b5ea64390078bceb596d8bc0f235c45cd470e44e6d09 734B / 734B                                                                            0.4s
 => => sha256:5777e64909dd0260c9960d3608ea5a7efa700aa288a8f1fcf014feaf54916206 162B / 162B                                                                            0.4s
 => => sha256:914e2f6f2525c162a2725753330257cdbc754d365cc55856402609d0dd047038 46.96MB / 46.96MB                                                                      4.0s
 => => sha256:7264a8db6415046d36d16ba98b79778e18accee6ffa71850405994cffa9be7de 3.40MB / 3.40MB                                                                        0.7s
 => => extracting sha256:7264a8db6415046d36d16ba98b79778e18accee6ffa71850405994cffa9be7de                                                                             0.2s
 => => extracting sha256:994e83f716a0db024fb37caf707eae5af27172a0fffb691c6e7b53bb7fc5b3ab                                                                             0.6s
 => => extracting sha256:914e2f6f2525c162a2725753330257cdbc754d365cc55856402609d0dd047038                                                                             1.4s
 => => extracting sha256:5777e64909dd0260c9960d3608ea5a7efa700aa288a8f1fcf014feaf54916206                                                                             0.0s
 => => extracting sha256:6a2bf8aa57bd3ac37dc6b5ea64390078bceb596d8bc0f235c45cd470e44e6d09                                                                             0.0s
 => => extracting sha256:5cf5f0762cf7c3135a3c63cc4711ca56eef959f0cfe5731febc20b271e09ff12                                                                             0.0s
 => => extracting sha256:041526fa6c45ea6a1845a0cbdc7edd6eefbe586959246af11fc0200d562b5816                                                                             2.2s
 => => extracting sha256:c2fb05fc177834cb351a4f6f3d4e204f9497b99bd9d1f812e7067bc185d9e82d                                                                             0.0s
 => => extracting sha256:86787f6d1cfdad7606d38054a7b28cad6ceabf6c13011c2876c1a7dff5278917                                                                             0.3s
 => => extracting sha256:3e34833d144bccde30401d665ae65ba571447a39a04c2cc02b10a81e8858ce9d                                                                             0.0s
 => => extracting sha256:539d3b93165bbb621c904215f176b90cf5854928fa446618f8af28821e78127a                                                                             0.0s
 => => extracting sha256:30e3f748b11ffca52e071d6ca397a39cc9ee191436a5e7fb8ea547c677cb7e41                                                                             1.3s
 => => extracting sha256:fe25c66a9a3122454dc7f33ecfc88233fc08ad6b49f9f35c0e1b476adc60f617                                                                             0.0s
 => => extracting sha256:00b38c0d6d7e62793aa6c4200f3c5e404949c34f8a18629ab5ecdec2cdf6a349                                                                             0.0s
 => => extracting sha256:170e4e13009e9b1d243e32f065068f72c223fbeafc045495da6c010b32a2ab95                                                                             0.5s
 => => extracting sha256:c9d7ead20553853875b36959215175c9a89c23ce08f0e17da02600a1b0e8a5c6                                                                             2.1s
 => apk update                                                                                                                                                        5.2s
 => apk add --no-cache curl libcurl                                                                                                                                   3.2s
 => apk upgrade --no-cache curl libcurl                                                                                                                               4.3s 
 => mkdir /copa-out                                                                                                                                                   0.0s 
 => sh -c apk info --installed -v curl libcurl > results.manifest; if [[ $? -ne 0 ]]; then echo "WARN: apk info --installed returned $?"; fi                          0.2s 
 => diff (apk upgrade --no-cache curl libcurl) -> (sh -c apk info --installed -v curl libcurl > results.manifest; if [[ $? -ne 0 ]]; then echo "WARN: apk info --ins  0.1s 
 => => merging                                                                                                                                                        0.1s 
 => exporting to client directory                                                                                                                                     0.1s 
 => => copying files 109B                                                                                                                                             0.0s
INFO[0036] Validated package curl version 8.3.0-r0 meets requested version 8.3.0-r0 
INFO[0036] Validated package libcurl version 8.3.0-r0 meets requested version 8.3.0-r0 
[+] Building 0.4s (7/7) FINISHED                                                                                                                                           
 => CACHED docker-image://my-image                                                                                        0.2s
 => => resolve my-image                                                                                                   0.2s
 => CACHED apk update                                                                                                                                                 0.0s
 => CACHED apk add --no-cache curl libcurl                                                                                                                            0.0s
 => CACHED apk upgrade --no-cache curl libcurl                                                                                                                        0.0s
 => diff (apk update) -> (apk upgrade --no-cache curl libcurl)                                                                                                        0.0s
 => merge (docker-image://my-image, diff (apk update) -> (apk upgrade --no-cache curl libcurl))                           0.0s
 => ERROR exporting to docker image format                                                                                                                            0.1s
 => => exporting layers                                                                                                                                               0.1s
 => => exporting manifest sha256:15ad5322a7e10486a6ca9865856372d0643607e03bd0906b64ec81cc8278382f                                                                     0.0s
 => => exporting config sha256:e1228a2a8dedabbc564cdefd337ffda61903c5925ef4956bd66f00d1fcbe0e76                                                                       0.0s
------
 > exporting to docker image format:
------

And it stuck here, till killed (I can add --timeout 1m, but it still failed on same error : ERROR exporting to docker image format) Any ideas will be appreciated.

sozercan commented 11 months ago

@lioryantov thanks for the report! can you provide the output with --debug?

can you also provide a similar dockerfile (where you can repro this) and/or image?

lioryantov commented 11 months ago

Sure, attached copa.json with trivy scan result.

$ trivy image --vuln-type os --ignore-unfixed -f json -o copa-patch-my-be.json myacr.azurecr.io/my-be-service:1.0.2.1504 

$ cat copa-patch-my-be.json 
{
  "SchemaVersion": 2,
  "ArtifactName": "myacr.azurecr.io/my-be-service:1.0.2.1504",
  "ArtifactType": "container_image",
  "Metadata": {
    "OS": {
      "Family": "alpine",
      "Name": "3.18.3"
    },
    "ImageID": "sha256:c0ffe1f2f6ce6802db8252c09dacc34bc83e70befaa9f98bcdd2f40af2056550",
    "DiffIDs": [
      "sha256:4693057ce2364720d39e57e85a5b8e0bd9ac3573716237736d6470ec5b7b7230",
      "sha256:1061c27d7cdf5af544ae24b937a7ddf5452a1eca7e673f4a30d091b3781aa42d",
      "sha256:7ecdf106796ffa1097a5b34d2fc1d2de1ad2b55a03f6165d29924d346e3864c3",
      "sha256:bcf631d599474f298fe12bc68d314c2fdac0459e79c61daf062d56793d1a99ad",
      "sha256:ad0ce4e8008d5d70efbd2c36e1597660d16f45d0308f6371064d0bb93924a17e",
      "sha256:729710009f76ee643b49defe6d9b600b8fb23e7bee490b8a6f43c8a84cf23170",
      "sha256:cec46aa8a964a05e8f3d575c4857602d380e7be760c345393ff548253e4cd4fb",
      "sha256:f3498642ab8f46103b3b8f4979b29ec86db812984448a93d6f454fb22ea6b9de",
      "sha256:d6c5a38ddc21daf8282b92ac7a71dc2737f4f3242a4cd9e2bdc7fdb9f006444c",
      "sha256:436ec8646e42eadf16bfbcff0b9ea8ebb7294314f2a6a7ab4b2e48feb0624c26",
      "sha256:293a085783a9961089669f37da84f42a2b0eb3c28bd1a078b9aebd00d8ff04a8",
      "sha256:80c33bf8a51fda46eceb036315dc6837f44168152eb34f8b19194a15201f89ec",
      "sha256:af4896456f11b7cde7c791ebd8ecbe55abe807167ae18f73daa8bdd6e7118361",
      "sha256:bffb362603638f5e4d2bedae4109ed6e8171522899a63224b29c376b49b85dc8",
      "sha256:2527f75b8f7541e6a7fbf22b2d6e6b50120a6ec94184fe8e1d4817b35f91ef7f",
      "sha256:40c272897e94d7edc03cf7b277d3f9ab79b69f3b41790fa3c1d24b126e60a8bd"
    ],
    "RepoTags": [
      "myacr.azurecr.io/my-be-service:1.0.2.1504"
    ],
    "RepoDigests": [
      "myacr.azurecr.io/my-be-service@sha256:7561b386bcada18c17f36e3f6daa7915c96417896fc185857c234c1a55746a38"
    ],
    "ImageConfig": {
      "architecture": "amd64",
      "created": "2023-09-21T14:27:36.669125519Z",
      "history": [
        {
          "created": "2023-08-07T19:20:20Z",
          "created_by": "/bin/sh -c #(nop) ADD file:32ff5e7a78b890996ee4681cc0a26185d3e9acdb4eb1e2aaccb2411f922fed6b in / "
        },
        {
          "created": "2023-08-07T19:20:20Z",
          "created_by": "/bin/sh -c #(nop)  CMD [\"/bin/sh\"]",
          "empty_layer": true
        },
        {
          "created": "2023-08-08T19:19:44Z",
          "created_by": "/bin/sh -c #(nop)  ENV JAVA_HOME=/opt/java/openjdk",
          "empty_layer": true
        },
        {
          "created": "2023-08-08T19:19:44Z",
          "created_by": "/bin/sh -c #(nop)  ENV PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
          "empty_layer": true
        },
        {
          "created": "2023-08-08T19:19:44Z",
          "created_by": "/bin/sh -c #(nop)  ENV LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8",
          "empty_layer": true
        },
        {
          "created": "2023-08-14T18:09:08Z",
          "created_by": "/bin/sh -c apk add --no-cache fontconfig java-cacerts bash libretls musl-locales musl-locales-lang ttf-dejavu tzdata zlib     \u0026\u0026 rm -rf /var/cache/apk/*"
        },
        {
          "created": "2023-08-31T20:23:30Z",
          "created_by": "/bin/sh -c #(nop)  ENV JAVA_VERSION=jdk-17.0.8.1+1",
          "empty_layer": true
        },
        {
          "created": "2023-08-31T20:24:50Z",
          "created_by": "/bin/sh -c set -eux;     ARCH=\"$(apk --print-arch)\";     case \"${ARCH}\" in        amd64|x86_64)          ESUM='bf726bb99785901f22849a0ef4ddd4e67f3e5b184dbbf260fffbaf5befce18a3';          BINARY_URL='https://github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.8.1%2B1/OpenJDK17U-jre_x64_alpine-linux_hotspot_17.0.8.1_1.tar.gz';          ;;        *)          echo \"Unsupported arch: ${ARCH}\";          exit 1;          ;;     esac; \t  wget -O /tmp/openjdk.tar.gz ${BINARY_URL}; \t  echo \"${ESUM} */tmp/openjdk.tar.gz\" | sha256sum -c -; \t  mkdir -p \"$JAVA_HOME\"; \t  tar --extract \t      --file /tmp/openjdk.tar.gz \t      --directory \"$JAVA_HOME\" \t      --strip-components 1 \t      --no-same-owner \t  ;     rm -f /tmp/openjdk.tar.gz ${JAVA_HOME}/lib/src.zip;"
        },
        {
          "created": "2023-08-31T20:24:51Z",
          "created_by": "/bin/sh -c echo Verifying install ...     \u0026\u0026 fileEncoding=\"$(echo 'System.out.println(System.getProperty(\"file.encoding\"))' | jshell -s -)\"; [ \"$fileEncoding\" = 'UTF-8' ]; rm -rf ~/.java     \u0026\u0026 echo java --version \u0026\u0026 java --version     \u0026\u0026 echo Complete."
        },
        {
          "created": "2023-08-31T20:24:51Z",
          "created_by": "/bin/sh -c #(nop) COPY file:8b8864b3e02a33a579dc216fd51b28a6047bc8eeaa03045b258980fe0cf7fcb3 in /__cacert_entrypoint.sh "
        },
        {
          "created": "2023-08-31T20:24:51Z",
          "created_by": "/bin/sh -c #(nop)  ENTRYPOINT [\"/__cacert_entrypoint.sh\"]",
          "empty_layer": true
        },
        {
          "created": "2023-09-18T02:12:14Z",
          "created_by": "ENV APP_PATH=/opt/app/nova-app",
          "comment": "buildkit.dockerfile.v0",
          "empty_layer": true
        },
        {
          "created": "2023-09-18T02:12:14Z",
          "created_by": "WORKDIR /opt/app/nova-app",
          "comment": "buildkit.dockerfile.v0"
        },
        {
          "created": "2023-09-21T14:27:31Z",
          "created_by": "RUN /bin/sh -c apk --update --no-cache add     bash     curl     vim     fontconfig     ttf-dejavu     mailx     nfs-utils     tshark     jattach     eudev # buildkit",
          "comment": "buildkit.dockerfile.v0"
        },
        {
          "created": "2023-09-21T14:27:31Z",
          "created_by": "ARG USER_NAME=novauser",
          "comment": "buildkit.dockerfile.v0",
          "empty_layer": true
        },
        {
          "created": "2023-09-21T14:27:31Z",
          "created_by": "ARG GROUP_NAME=novagroup",
          "comment": "buildkit.dockerfile.v0",
          "empty_layer": true
        },
        {
          "created": "2023-09-21T14:27:31Z",
          "created_by": "RUN |2 USER_NAME=novauser GROUP_NAME=novagroup /bin/sh -c addgroup -g 901 -S $GROUP_NAME     \u0026\u0026 adduser -u 900 -S $USER_NAME -G $GROUP_NAME     \u0026\u0026 chown $USER_NAME:$GROUP_NAME ./ # buildkit",
          "comment": "buildkit.dockerfile.v0"
        },
        {
          "created": "2023-09-21T14:27:31Z",
          "created_by": "COPY ./applicationinsights-agent-3.4.15.jar /opt/app/nova-app/applicationinsights-agent-3.4.15.jar # buildkit",
          "comment": "buildkit.dockerfile.v0"
        },
        {
          "created": "2023-09-21T14:27:31Z",
          "created_by": "CMD [\"/bin/sh\" \"-c\" \"true\"]",
          "comment": "buildkit.dockerfile.v0",
          "empty_layer": true
        },
        {
          "created": "2023-09-21T14:27:31Z",
          "created_by": "COPY ALM/AZ/applicationinsights.json /opt/app/nova-app/applicationinsights.json # buildkit",
          "comment": "buildkit.dockerfile.v0"
        },
        {
          "created": "2023-09-21T14:27:31Z",
          "created_by": "CMD [\"/bin/sh\" \"-c\" \"true\"]",
          "comment": "buildkit.dockerfile.v0",
          "empty_layer": true
        },
        {
          "created": "2023-09-21T14:27:31Z",
          "created_by": "COPY ./nextVersion.properties /opt/app/nova-app # buildkit",
          "comment": "buildkit.dockerfile.v0"
        },
        {
          "created": "2023-09-21T14:27:31Z",
          "created_by": "CMD [\"/bin/sh\" \"-c\" \"true\"]",
          "comment": "buildkit.dockerfile.v0",
          "empty_layer": true
        },
        {
          "created": "2023-09-21T14:27:34Z",
          "created_by": "COPY application/dependencies/ ./ # buildkit",
          "comment": "buildkit.dockerfile.v0"
        },
        {
          "created": "2023-09-21T14:27:34Z",
          "created_by": "CMD [\"/bin/sh\" \"-c\" \"true\"]",
          "comment": "buildkit.dockerfile.v0",
          "empty_layer": true
        },
        {
          "created": "2023-09-21T14:27:34Z",
          "created_by": "COPY application/spring-boot-loader/ ./ # buildkit",
          "comment": "buildkit.dockerfile.v0"
        },
        {
          "created": "2023-09-21T14:27:34Z",
          "created_by": "CMD [\"/bin/sh\" \"-c\" \"true\"]",
          "comment": "buildkit.dockerfile.v0",
          "empty_layer": true
        },
        {
          "created": "2023-09-21T14:27:34Z",
          "created_by": "COPY application/snapshot-dependencies/ ./ # buildkit",
          "comment": "buildkit.dockerfile.v0"
        },
        {
          "created": "2023-09-21T14:27:34Z",
          "created_by": "CMD [\"/bin/sh\" \"-c\" \"true\"]",
          "comment": "buildkit.dockerfile.v0",
          "empty_layer": true
        },
        {
          "created": "2023-09-21T14:27:34Z",
          "created_by": "COPY application/application/ ./ # buildkit",
          "comment": "buildkit.dockerfile.v0"
        },
        {
          "created": "2023-09-21T14:27:34Z",
          "created_by": "CMD [\"/bin/sh\" \"-c\" \"true\"]",
          "comment": "buildkit.dockerfile.v0",
          "empty_layer": true
        },
        {
          "created": "2023-09-21T14:27:36Z",
          "created_by": "RUN |2 USER_NAME=novauser GROUP_NAME=novagroup /bin/sh -c chmod -R 777 ${APP_PATH} # buildkit",
          "comment": "buildkit.dockerfile.v0"
        },
        {
          "created": "2023-09-21T14:27:36Z",
          "created_by": "USER novauser",
          "comment": "buildkit.dockerfile.v0",
          "empty_layer": true
        },
        {
          "created": "2023-09-21T14:27:36Z",
          "created_by": "ENTRYPOINT [\"java\" \"-XX:NativeMemoryTracking=detail\" \"org.springframework.boot.loader.JarLauncher\"]",
          "comment": "buildkit.dockerfile.v0",
          "empty_layer": true
        }
      ],
      "os": "linux",
      "rootfs": {
        "type": "layers",
        "diff_ids": [
          "sha256:4693057ce2364720d39e57e85a5b8e0bd9ac3573716237736d6470ec5b7b7230",
          "sha256:1061c27d7cdf5af544ae24b937a7ddf5452a1eca7e673f4a30d091b3781aa42d",
          "sha256:7ecdf106796ffa1097a5b34d2fc1d2de1ad2b55a03f6165d29924d346e3864c3",
          "sha256:bcf631d599474f298fe12bc68d314c2fdac0459e79c61daf062d56793d1a99ad",
          "sha256:ad0ce4e8008d5d70efbd2c36e1597660d16f45d0308f6371064d0bb93924a17e",
          "sha256:729710009f76ee643b49defe6d9b600b8fb23e7bee490b8a6f43c8a84cf23170",
          "sha256:cec46aa8a964a05e8f3d575c4857602d380e7be760c345393ff548253e4cd4fb",
          "sha256:f3498642ab8f46103b3b8f4979b29ec86db812984448a93d6f454fb22ea6b9de",
          "sha256:d6c5a38ddc21daf8282b92ac7a71dc2737f4f3242a4cd9e2bdc7fdb9f006444c",
          "sha256:436ec8646e42eadf16bfbcff0b9ea8ebb7294314f2a6a7ab4b2e48feb0624c26",
          "sha256:293a085783a9961089669f37da84f42a2b0eb3c28bd1a078b9aebd00d8ff04a8",
          "sha256:80c33bf8a51fda46eceb036315dc6837f44168152eb34f8b19194a15201f89ec",
          "sha256:af4896456f11b7cde7c791ebd8ecbe55abe807167ae18f73daa8bdd6e7118361",
          "sha256:bffb362603638f5e4d2bedae4109ed6e8171522899a63224b29c376b49b85dc8",
          "sha256:2527f75b8f7541e6a7fbf22b2d6e6b50120a6ec94184fe8e1d4817b35f91ef7f",
          "sha256:40c272897e94d7edc03cf7b277d3f9ab79b69f3b41790fa3c1d24b126e60a8bd"
        ]
      },
      "config": {
        "Cmd": [
          "/bin/sh",
          "-c",
          "true"
        ],
        "Entrypoint": [
          "java",
          "-XX:NativeMemoryTracking=detail",
          "org.springframework.boot.loader.JarLauncher"
        ],
        "Env": [
          "PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
          "JAVA_HOME=/opt/java/openjdk",
          "LANG=en_US.UTF-8",
          "LANGUAGE=en_US:en",
          "LC_ALL=en_US.UTF-8",
          "JAVA_VERSION=jdk-17.0.8.1+1",
          "APP_PATH=/opt/app/nova-app"
        ],
        "User": "novauser",
        "WorkingDir": "/opt/app/nova-app",
        "ArgsEscaped": true
      }
    }
  },
  "Results": [
    {
      "Target": "myacr.azurecr.io/my-be-service:1.0.2.1504 (alpine 3.18.3)",
      "Class": "os-pkgs",
      "Type": "alpine",
      "Vulnerabilities": [
        {
          "VulnerabilityID": "CVE-2023-38039",
          "PkgID": "curl@8.2.1-r0",
          "PkgName": "curl",
          "InstalledVersion": "8.2.1-r0",
          "FixedVersion": "8.3.0-r0",
          "Status": "fixed",
          "Layer": {
            "DiffID": "sha256:cec46aa8a964a05e8f3d575c4857602d380e7be760c345393ff548253e4cd4fb"
          },
          "SeveritySource": "nvd",
          "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2023-38039",
          "DataSource": {
            "ID": "alpine",
            "Name": "Alpine Secdb",
            "URL": "https://secdb.alpinelinux.org/"
          },
          "Title": "out of heap memory issue due to missing limit on header quantity",
          "Description": "When curl retrieves an HTTP response, it stores the incoming headers so that\nthey can be accessed later via the libcurl headers API.\n\nHowever, curl did not have a limit in how many or how large headers it would\naccept in a response, allowing a malicious server to stream an endless series\nof headers and eventually cause curl to run out of heap memory.",
          "Severity": "HIGH",
          "CweIDs": [
            "CWE-770"
          ],
          "CVSS": {
            "nvd": {
              "V3Vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H",
              "V3Score": 7.5
            },
            "redhat": {
              "V3Vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H",
              "V3Score": 7.5
            }
          },
          "References": [
            "https://access.redhat.com/security/cve/CVE-2023-38039",
            "https://curl.se/docs/CVE-2023-38039.html",
            "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-38039",
            "https://hackerone.com/reports/2072338",
            "https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/5DCZMYODALBLVOXVJEN2LF2MLANEYL4F/",
            "https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/M6KGKB2JNZVT276JYSKI6FV2VFJUGDOJ/",
            "https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/TEAWTYHC3RT6ZRS5OZRHLAIENVN6CCIS/",
            "https://nvd.nist.gov/vuln/detail/CVE-2023-38039",
            "https://ubuntu.com/security/notices/USN-6363-1",
            "https://www.cve.org/CVERecord?id=CVE-2023-38039"
          ],
          "PublishedDate": "2023-09-15T04:15:00Z",
          "LastModifiedDate": "2023-09-27T15:18:00Z"
        },
        {
          "VulnerabilityID": "CVE-2023-38039",
          "PkgID": "libcurl@8.2.1-r0",
          "PkgName": "libcurl",
          "InstalledVersion": "8.2.1-r0",
          "FixedVersion": "8.3.0-r0",
          "Status": "fixed",
          "Layer": {
            "DiffID": "sha256:cec46aa8a964a05e8f3d575c4857602d380e7be760c345393ff548253e4cd4fb"
          },
          "SeveritySource": "nvd",
          "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2023-38039",
          "DataSource": {
            "ID": "alpine",
            "Name": "Alpine Secdb",
            "URL": "https://secdb.alpinelinux.org/"
          },
          "Title": "out of heap memory issue due to missing limit on header quantity",
          "Description": "When curl retrieves an HTTP response, it stores the incoming headers so that\nthey can be accessed later via the libcurl headers API.\n\nHowever, curl did not have a limit in how many or how large headers it would\naccept in a response, allowing a malicious server to stream an endless series\nof headers and eventually cause curl to run out of heap memory.",
          "Severity": "HIGH",
          "CweIDs": [
            "CWE-770"
          ],
          "CVSS": {
            "nvd": {
              "V3Vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H",
              "V3Score": 7.5
            },
            "redhat": {
              "V3Vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H",
              "V3Score": 7.5
            }
          },
          "References": [
            "https://access.redhat.com/security/cve/CVE-2023-38039",
            "https://curl.se/docs/CVE-2023-38039.html",
            "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-38039",
            "https://hackerone.com/reports/2072338",
            "https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/5DCZMYODALBLVOXVJEN2LF2MLANEYL4F/",
            "https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/M6KGKB2JNZVT276JYSKI6FV2VFJUGDOJ/",
            "https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/TEAWTYHC3RT6ZRS5OZRHLAIENVN6CCIS/",
            "https://nvd.nist.gov/vuln/detail/CVE-2023-38039",
            "https://ubuntu.com/security/notices/USN-6363-1",
            "https://www.cve.org/CVERecord?id=CVE-2023-38039"
          ],
          "PublishedDate": "2023-09-15T04:15:00Z",
          "LastModifiedDate": "2023-09-27T15:18:00Z"
        }
      ]
    }
  ]
}

Attached copa output with "--debug" flag.

$ copa patch -i myacr.azurecr.io/my-be-service:1.0.2.1504 -r copa-patch-my-be.json -t myacr.azurecr.io/my-be-service:1.0.2.1504-patched -a tcp://0.0.0.0:8888 --debug --timeout 1m
DEBU[0000] updates to apply: &{alpine 3.18.3 amd64 [{curl 8.3.0-r0} {libcurl 8.3.0-r0}]} 
DEBU[0000] resolving                                     host=myacr.azurecr.io
DEBU[0000] do request                                    host=myacr.azurecr.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0-dev request.method=HEAD url="https://myacr.azurecr.io/v2/my-be-service/manifests/1.0.2.1504"
DEBU[0000] fetch response received                       host=myacr.azurecr.io response.header.access-control-expose-headers=Docker-Content-Digest response.header.access-control-expose-headers.1=WWW-Authenticate response.header.access-control-expose-headers.2=Link response.header.access-control-expose-headers.3=X-Ms-Correlation-Request-Id response.header.connection=keep-alive response.header.content-length=209 response.header.content-type="application/json; charset=utf-8" response.header.date="Mon, 02 Oct 2023 19:49:47 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.server=openresty response.header.strict-transport-security="max-age=31536000; includeSubDomains" response.header.strict-transport-security.1="max-age=31536000; includeSubDomains" response.header.www-authenticate="Bearer realm=\"https://myacr.azurecr.io/oauth2/token\",service=\"myacr.azurecr.io\",scope=\"repository:my-be-service:pull\"" response.header.x-content-type-options=nosniff response.header.x-ms-correlation-request-id=b0c3bdb4-aaec-4757-b1dd-306657fc7f64 response.status="401 Unauthorized" url="https://myacr.azurecr.io/v2/my-be-service/manifests/1.0.2.1504"
DEBU[0000] Unauthorized                                  header="Bearer realm=\"https://myacr.azurecr.io/oauth2/token\",service=\"myacr.azurecr.io\",scope=\"repository:my-be-service:pull\"" host=myacr.azurecr.io
DEBU[0000] do request                                    host=myacr.azurecr.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0-dev request.method=HEAD url="https://myacr.azurecr.io/v2/my-be-service/manifests/1.0.2.1504"
DEBU[0001] fetch response received                       host=myacr.azurecr.io response.header.access-control-expose-headers=Docker-Content-Digest response.header.access-control-expose-headers.1=WWW-Authenticate response.header.access-control-expose-headers.2=Link response.header.access-control-expose-headers.3=X-Ms-Correlation-Request-Id response.header.connection=keep-alive response.header.content-length=3674 response.header.content-type=application/vnd.docker.distribution.manifest.v2+json response.header.date="Mon, 02 Oct 2023 19:49:47 GMT" response.header.docker-content-digest="sha256:7561b386bcada18c17f36e3f6daa7915c96417896fc185857c234c1a55746a38" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:7561b386bcada18c17f36e3f6daa7915c96417896fc185857c234c1a55746a38\"" response.header.server=openresty response.header.strict-transport-security="max-age=31536000; includeSubDomains" response.header.strict-transport-security.1="max-age=31536000; includeSubDomains" response.header.x-content-type-options=nosniff response.header.x-ms-client-request-id= response.header.x-ms-correlation-request-id=93ece010-50bc-42cd-8eee-498fb4ed8d8f response.header.x-ms-request-id=283c2f74-367b-4bc7-a841-0d41280844c6 response.status="200 OK" url="https://myacr.azurecr.io/v2/my-be-service/manifests/1.0.2.1504"
DEBU[0001] resolved                                      desc.digest="sha256:7561b386bcada18c17f36e3f6daa7915c96417896fc185857c234c1a55746a38" host=myacr.azurecr.io
DEBU[0001] fetch                                         digest="sha256:7561b386bcada18c17f36e3f6daa7915c96417896fc185857c234c1a55746a38" mediatype=application/vnd.docker.distribution.manifest.v2+json size=3674
DEBU[0001] do request                                    digest="sha256:7561b386bcada18c17f36e3f6daa7915c96417896fc185857c234c1a55746a38" mediatype=application/vnd.docker.distribution.manifest.v2+json request.header.accept="application/vnd.docker.distribution.manifest.v2+json, */*" request.header.user-agent=buildkit/v0.0-dev request.method=GET size=3674 url="https://myacr.azurecr.io/v2/my-be-service/manifests/sha256:7561b386bcada18c17f36e3f6daa7915c96417896fc185857c234c1a55746a38"
DEBU[0001] fetch response received                       digest="sha256:7561b386bcada18c17f36e3f6daa7915c96417896fc185857c234c1a55746a38" mediatype=application/vnd.docker.distribution.manifest.v2+json response.header.access-control-expose-headers=Docker-Content-Digest response.header.access-control-expose-headers.1=WWW-Authenticate response.header.access-control-expose-headers.2=Link response.header.access-control-expose-headers.3=X-Ms-Correlation-Request-Id response.header.connection=keep-alive response.header.content-length=3674 response.header.content-type=application/vnd.docker.distribution.manifest.v2+json response.header.date="Mon, 02 Oct 2023 19:49:47 GMT" response.header.docker-content-digest="sha256:7561b386bcada18c17f36e3f6daa7915c96417896fc185857c234c1a55746a38" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:7561b386bcada18c17f36e3f6daa7915c96417896fc185857c234c1a55746a38\"" response.header.server=openresty response.header.strict-transport-security="max-age=31536000; includeSubDomains" response.header.strict-transport-security.1="max-age=31536000; includeSubDomains" response.header.x-content-type-options=nosniff response.header.x-ms-client-request-id= response.header.x-ms-correlation-request-id=75e3acee-8152-4d45-8db8-721122ec195b response.header.x-ms-request-id=3397ae1a-88ce-493a-8858-8a976433c385 response.status="200 OK" size=3674 url="https://myacr.azurecr.io/v2/my-be-service/manifests/sha256:7561b386bcada18c17f36e3f6daa7915c96417896fc185857c234c1a55746a38"
DEBU[0001] fetch                                         digest="sha256:c0ffe1f2f6ce6802db8252c09dacc34bc83e70befaa9f98bcdd2f40af2056550" mediatype=application/vnd.docker.container.image.v1+json size=8218
DEBU[0001] do request                                    digest="sha256:c0ffe1f2f6ce6802db8252c09dacc34bc83e70befaa9f98bcdd2f40af2056550" mediatype=application/vnd.docker.container.image.v1+json request.header.accept="application/vnd.docker.container.image.v1+json, */*" request.header.user-agent=buildkit/v0.0-dev request.method=GET size=8218 url="https://myacr.azurecr.io/v2/my-be-service/blobs/sha256:c0ffe1f2f6ce6802db8252c09dacc34bc83e70befaa9f98bcdd2f40af2056550"
DEBU[0003] fetch response received                       digest="sha256:c0ffe1f2f6ce6802db8252c09dacc34bc83e70befaa9f98bcdd2f40af2056550" mediatype=application/vnd.docker.container.image.v1+json response.header.accept-ranges=bytes response.header.connection=keep-alive response.header.content-length=8218 response.header.content-type=application/octet-stream response.header.date="Mon, 02 Oct 2023 19:49:49 GMT" response.header.etag="\"0x8DBBBAB326DBFA2\"" response.header.last-modified="Fri, 22 Sep 2023 20:33:36 GMT" response.header.server=openresty response.header.x-ms-blob-committed-block-count=1 response.header.x-ms-blob-type=AppendBlob response.header.x-ms-copy-completion-time="Fri, 22 Sep 2023 20:33:36 GMT" response.header.x-ms-copy-id=25fa679b-9771-4c53-8797-6dff32a6c942 response.header.x-ms-copy-progress=8218/8218 response.header.x-ms-copy-status=success response.header.x-ms-creation-time="Fri, 22 Sep 2023 20:33:35 GMT" response.header.x-ms-lease-state=available response.header.x-ms-lease-status=unlocked response.header.x-ms-request-id=49463098-001e-006b-4369-f56f04000000 response.header.x-ms-server-encrypted=true response.header.x-ms-version=2018-03-28 response.status="200 OK" size=8218 url="https://myacr.azurecr.io/v2/my-be-service/blobs/sha256:c0ffe1f2f6ce6802db8252c09dacc34bc83e70befaa9f98bcdd2f40af2056550"
DEBU[0003] latest unique APKs: [{curl 8.3.0-r0} {libcurl 8.3.0-r0}] 
DEBU[0003] serving grpc connection                      
[+] Building 1.1s (0/7)                                                                                                                                                    
[+] Building 1.2s (8/8) FINISHED                                                                                                                                           
 => docker-image://myacr.azurecr.io/my-be-service:1.0.2.1504                                                                                               1.1s
 => => resolve myacr.azurecr.io/my-be-service:1.0.2.1504                                                                                                   1.1s
 => CACHED apk update                                                                                                                                                 0.0s
 => CACHED apk add --no-cache curl libcurl                                                                                                                            0.0s
 => CACHED apk upgrade --no-cache curl libcurl                                                                                                                        0.0s
 => CACHED mkdir /copa-out                                                                                                                                            0.0s
 => CACHED sh -c apk info --installed -v curl libcurl > results.manifest; if [[ $? -ne 0 ]]; then echo "WARN: apk info --installed returned $?"; fi                   0.0s
 => CACHED diff (apk upgrade --no-cache curl libcurl) -> (sh -c apk info --installed -v curl libcurl > results.manifest; if [[ $? -ne 0 ]]; then echo "WARN: apk inf  0.0s
 => exporting to client directory                                                                                                                                     0.0s
 => => copying files 109B                                                                                                                                             0.0s
DEBU[0004] Wrote LLB state to /tmp/copa-264221316       
DEBU[0004] Required updates: [{curl 8.3.0-r0} {libcurl 8.3.0-r0}] 
DEBU[0004] Resulting updates: [curl-8.3.0-r0 libcurl-8.3.0-r0] 
INFO[0004] Validated package curl version 8.3.0-r0 meets requested version 8.3.0-r0 
INFO[0004] Validated package libcurl version 8.3.0-r0 meets requested version 8.3.0-r0 
DEBU[0004] serving grpc connection                      
[+] Building 0.1s (0/6)                                                                                                                                                    
[+] Building 0.3s (7/7) FINISHED                                                                                                                                           
 => docker-image://myacr.azurecr.io/my-be-service:1.0.2.1504                                                                                               0.2s
 => => resolve myacr.azurecr.io/my-be-service:1.0.2.1504                                                                                                   0.2s
 => CACHED apk update                                                                                                                                                 0.0s
 => CACHED apk add --no-cache curl libcurl                                                                                                                            0.0s
 => CACHED apk upgrade --no-cache curl libcurl                                                                                                                        0.0s
 => CACHED diff (apk update) -> (apk upgrade --no-cache curl libcurl)                                                                                                 0.0s
 => CACHED merge (docker-image://myacr.azurecr.io/my-be-service:1.0.2.1504, diff (apk update) -> (apk upgrade --no-cache curl libcurl))                    0.0s
 => ERROR exporting to docker image format                                                                                                                            0.0s
 => => exporting layers                                                                                                                                               0.0s
 => => exporting manifest sha256:15ad5322a7e10486a6ca9865856372d0643607e03bd0906b64ec81cc8278382f                                                                     0.0s
 => => exporting config sha256:e1228a2a8dedabbc564cdefd337ffda61903c5925ef4956bd66f00d1fcbe0e76                                                                       0.0s
------
 > exporting to docker image format:
------
ERRO[0061] patch exceeded timeout 1m0s                  
Error: patch exceeded timeout 1m0s

Dockerfile of this "my-be" microservice looks like that:

FROM eclipse-temurin:17-jre-alpine as builder

WORKDIR application
COPY target/my-be-*.jar application.jar
RUN java -Djarmode=layertools -jar application.jar extract
RUN find .

FROM eclipse-temurin:17-jre-alpine

ENV APP_PATH=/opt/app/my-app

WORKDIR ${APP_PATH}

RUN apk --update --no-cache add \
    bash \
    curl \
    vim \
    fontconfig \
    ttf-dejavu \
    mailx \
    nfs-utils \
    tshark \
    jattach \
    eudev

ARG USER_NAME=myuser
ARG GROUP_NAME=mygroup
RUN addgroup -g 901 -S $GROUP_NAME \
    && adduser -u 900 -S $USER_NAME -G $GROUP_NAME \
    && chown $USER_NAME:$GROUP_NAME ./

COPY --from=builder --chown=$USER_NAME:$GROUP_NAME application/dependencies/ ./
COPY --from=builder --chown=$USER_NAME:$GROUP_NAME application/spring-boot-loader/ ./
COPY --from=builder --chown=$USER_NAME:$GROUP_NAME application/snapshot-dependencies/ ./
COPY --from=builder --chown=$USER_NAME:$GROUP_NAME application/application/ ./

RUN chmod -R 777 ${APP_PATH}

USER $USER_NAME

ENTRYPOINT ["java", "-XX:NativeMemoryTracking=detail", "org.springframework.boot.loader.JarLauncher"]
lioryantov commented 11 months ago

Hi, Any update?

ashnamehrotra commented 11 months ago

@lioryantov can you try using a different patched tag? Copa appends the tag to the image name so instead of -t myacr.azurecr.io/my-be-service:1.0.2.1504-patched you can use -t 1.0.2.1504-patched as the argument to Copa.

lioryantov commented 11 months ago

@lioryantov can you try using a different patched tag? Copa appends the tag to the image name so instead of -t myacr.azurecr.io/my-be-service:1.0.2.1504-patched you can use -t 1.0.2.1504-patched as the argument to Copa.

Thank you @ashnamehrotra the provided advise of using "-t 1.0.2.1504-patched" worked for this image. But when I am trying to patch other one, got following error:

$ copa patch -i myacr.azurecr.io/my-fe-service:1.0.1.985 -r copa-patch-nova-fe.json -t 1.0.2.985-patched -a tcp://0.0.0.0:8888 --debug --timeout 1m
DEBU[0000] updates to apply: &{debian 12.1 amd64 [{curl 7.88.1-10+deb12u3} {libc-bin 2.36-9+deb12u3} {libc-bin 2.36-9+deb12u3} {libc-bin 2.36-9+deb12u3} {libc6 2.36-9+deb12u3} {libc6 2.36-9+deb12u3} {libc6 2.36-9+deb12u3} {libcurl4 7.88.1-10+deb12u3} {libgssapi-krb5-2 1.20.1-2+deb12u1} {libk5crypto3 1.20.1-2+deb12u1} {libkrb5-3 1.20.1-2+deb12u1} {libkrb5support0 1.20.1-2+deb12u1} {libssl3 3.0.10-1~deb12u1} {libssl3 3.0.10-1~deb12u1} {libssl3 3.0.10-1~deb12u1} {libx11-6 2:1.8.4-2+deb12u2} {libx11-6 2:1.8.4-2+deb12u2} {libx11-6 2:1.8.4-2+deb12u2} {libx11-data 2:1.8.4-2+deb12u2} {libx11-data 2:1.8.4-2+deb12u2} {libx11-data 2:1.8.4-2+deb12u2} {libxpm4 1:3.5.12-1.1+deb12u1} {libxpm4 1:3.5.12-1.1+deb12u1} {openssl 3.0.10-1~deb12u1} {openssl 3.0.10-1~deb12u1} {openssl 3.0.10-1~deb12u1}]} 
DEBU[0000] resolving                                     host=myacr.azurecr.io
DEBU[0000] do request                                    host=myacr.azurecr.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0-dev request.method=HEAD url="https://myacr.azurecr.io/v2/my-fe-service/manifests/1.0.1.985"
DEBU[0000] fetch response received                       host=myacr.azurecr.io response.header.access-control-expose-headers=Docker-Content-Digest response.header.access-control-expose-headers.1=WWW-Authenticate response.header.access-control-expose-headers.2=Link response.header.access-control-expose-headers.3=X-Ms-Correlation-Request-Id response.header.connection=keep-alive response.header.content-length=209 response.header.content-type="application/json; charset=utf-8" response.header.date="Wed, 11 Oct 2023 18:07:20 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.server=openresty response.header.strict-transport-security="max-age=31536000; includeSubDomains" response.header.strict-transport-security.1="max-age=31536000; includeSubDomains" response.header.www-authenticate="Bearer realm=\"https://myacr.azurecr.io/oauth2/token\",service=\"myacr.azurecr.io\",scope=\"repository:my-fe-service:pull\"" response.header.x-content-type-options=nosniff response.header.x-ms-correlation-request-id=1aca2f0e-83d5-416c-a377-e2ef07fd79d7 response.status="401 Unauthorized" url="https://myacr.azurecr.io/v2/my-fe-service/manifests/1.0.1.985"
DEBU[0000] Unauthorized                                  header="Bearer realm=\"https://myacr.azurecr.io/oauth2/token\",service=\"myacr.azurecr.io\",scope=\"repository:my-fe-service:pull\"" host=myacr.azurecr.io
DEBU[0000] do request                                    host=myacr.azurecr.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0-dev request.method=HEAD url="https://myacr.azurecr.io/v2/my-fe-service/manifests/1.0.1.985"
DEBU[0001] fetch response received                       host=myacr.azurecr.io response.header.access-control-expose-headers=Docker-Content-Digest response.header.access-control-expose-headers.1=WWW-Authenticate response.header.access-control-expose-headers.2=Link response.header.access-control-expose-headers.3=X-Ms-Correlation-Request-Id response.header.connection=keep-alive response.header.content-length=5528 response.header.content-type=application/vnd.docker.distribution.manifest.v2+json response.header.date="Wed, 11 Oct 2023 18:07:21 GMT" response.header.docker-content-digest="sha256:03bfc8d353aef010ab4bc697ed5432cbc25548d5f7409a8d7625f80658c2c3e4" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:03bfc8d353aef010ab4bc697ed5432cbc25548d5f7409a8d7625f80658c2c3e4\"" response.header.server=openresty response.header.strict-transport-security="max-age=31536000; includeSubDomains" response.header.strict-transport-security.1="max-age=31536000; includeSubDomains" response.header.x-content-type-options=nosniff response.header.x-ms-client-request-id= response.header.x-ms-correlation-request-id=7c941516-98b9-4753-a083-bc01c3825d52 response.header.x-ms-request-id=92134a07-be63-4c20-a30f-c2d9bc5e60e3 response.status="200 OK" url="https://myacr.azurecr.io/v2/my-fe-service/manifests/1.0.1.985"
DEBU[0001] resolved                                      desc.digest="sha256:03bfc8d353aef010ab4bc697ed5432cbc25548d5f7409a8d7625f80658c2c3e4" host=myacr.azurecr.io
DEBU[0001] fetch                                         digest="sha256:03bfc8d353aef010ab4bc697ed5432cbc25548d5f7409a8d7625f80658c2c3e4" mediatype=application/vnd.docker.distribution.manifest.v2+json size=5528
DEBU[0001] do request                                    digest="sha256:03bfc8d353aef010ab4bc697ed5432cbc25548d5f7409a8d7625f80658c2c3e4" mediatype=application/vnd.docker.distribution.manifest.v2+json request.header.accept="application/vnd.docker.distribution.manifest.v2+json, */*" request.header.user-agent=buildkit/v0.0-dev request.method=GET size=5528 url="https://myacr.azurecr.io/v2/my-fe-service/manifests/sha256:03bfc8d353aef010ab4bc697ed5432cbc25548d5f7409a8d7625f80658c2c3e4"
DEBU[0001] fetch response received                       digest="sha256:03bfc8d353aef010ab4bc697ed5432cbc25548d5f7409a8d7625f80658c2c3e4" mediatype=application/vnd.docker.distribution.manifest.v2+json response.header.access-control-expose-headers=Docker-Content-Digest response.header.access-control-expose-headers.1=WWW-Authenticate response.header.access-control-expose-headers.2=Link response.header.access-control-expose-headers.3=X-Ms-Correlation-Request-Id response.header.connection=keep-alive response.header.content-length=5528 response.header.content-type=application/vnd.docker.distribution.manifest.v2+json response.header.date="Wed, 11 Oct 2023 18:07:21 GMT" response.header.docker-content-digest="sha256:03bfc8d353aef010ab4bc697ed5432cbc25548d5f7409a8d7625f80658c2c3e4" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:03bfc8d353aef010ab4bc697ed5432cbc25548d5f7409a8d7625f80658c2c3e4\"" response.header.server=openresty response.header.strict-transport-security="max-age=31536000; includeSubDomains" response.header.strict-transport-security.1="max-age=31536000; includeSubDomains" response.header.x-content-type-options=nosniff response.header.x-ms-client-request-id= response.header.x-ms-correlation-request-id=9ce781ff-b063-4b61-8b97-1c8e6d8e401d response.header.x-ms-request-id=a4acfb03-4d1a-462e-b1c4-e76130057575 response.status="200 OK" size=5528 url="https://myacr.azurecr.io/v2/my-fe-service/manifests/sha256:03bfc8d353aef010ab4bc697ed5432cbc25548d5f7409a8d7625f80658c2c3e4"
DEBU[0001] fetch                                         digest="sha256:9aea4e30edbc561987419c6d02a58953f6977420759116dd90910fdc55e04f4b" mediatype=application/vnd.docker.container.image.v1+json size=13347
DEBU[0001] do request                                    digest="sha256:9aea4e30edbc561987419c6d02a58953f6977420759116dd90910fdc55e04f4b" mediatype=application/vnd.docker.container.image.v1+json request.header.accept="application/vnd.docker.container.image.v1+json, */*" request.header.user-agent=buildkit/v0.0-dev request.method=GET size=13347 url="https://myacr.azurecr.io/v2/my-fe-service/blobs/sha256:9aea4e30edbc561987419c6d02a58953f6977420759116dd90910fdc55e04f4b"
DEBU[0002] fetch response received                       digest="sha256:9aea4e30edbc561987419c6d02a58953f6977420759116dd90910fdc55e04f4b" mediatype=application/vnd.docker.container.image.v1+json response.header.accept-ranges=bytes response.header.connection=keep-alive response.header.content-length=13347 response.header.content-type=application/octet-stream response.header.date="Wed, 11 Oct 2023 18:07:22 GMT" response.header.etag="\"0x8DBBFD5CF390FD0\"" response.header.last-modified="Thu, 28 Sep 2023 03:48:42 GMT" response.header.server=openresty response.header.x-ms-blob-committed-block-count=1 response.header.x-ms-blob-type=AppendBlob response.header.x-ms-copy-completion-time="Thu, 28 Sep 2023 03:48:42 GMT" response.header.x-ms-copy-id=d7dd91bb-2544-429a-8ebe-1a5e52cb84be response.header.x-ms-copy-progress=13347/13347 response.header.x-ms-copy-status=success response.header.x-ms-creation-time="Thu, 28 Sep 2023 03:48:42 GMT" response.header.x-ms-lease-state=available response.header.x-ms-lease-status=unlocked response.header.x-ms-request-id=531a397a-201e-0006-2d6d-fcd14f000000 response.header.x-ms-server-encrypted=true response.header.x-ms-version=2018-03-28 response.status="200 OK" size=13347 url="https://myacr.azurecr.io/v2/my-fe-service/blobs/sha256:9aea4e30edbc561987419c6d02a58953f6977420759116dd90910fdc55e04f4b"
DEBU[0002] Using debian:12-slim as basis for tooling image 
DEBU[0002] serving grpc connection                      
[+] Building 10.9s (2/8)                                                                                                                                                   
[+] Building 11.0s (5/8)                                                                                                                                                   
 => ERROR docker-image://docker.io/library/debian:12-slim                                                                                                            10.3s
 => => resolve docker.io/library/debian:12-slim                                                                                                                      10.3s
 => docker-image://myacr.azurecr.io/my-fe-service:1.0.1.985                                                                                               10.9s
 => => resolve myacr.azurecr.io/my-fe-service:1.0.1.985                                                                                                    0.9s
 => => extracting sha256:8b625c47d69711d95708566cd97b72bca565679d034ee0372e2e23499a0112be                                                                             1.1s
 => => sha256:42b973a49132d6e481157b7c3d8194dee2223e5c419b4fac7f4ad4b4530e4f5b 123.73MB / 292.00MB                                                                   10.0s
 => => sha256:4f8754d11d9d0f5c618323b7aa951a8b8de0f3fa5f56dd951682c4139445dc86 114.29MB / 292.01MB                                                                   10.0s
 => => sha256:2cab533ae9e08af8d5d70fff80c1e96ef0365b8c29e3ee93bcebba553c856a62 110.10MB / 292.00MB                                                                   10.0s
 => => extracting sha256:4d3239651a63f0595b1c047313d6f5c63e1e69c834d315dce09e2c092c2fcea7                                                                             0.0s
 => => extracting sha256:0f816efa513d909851c457ae41744fe3ff36ab19ebc2d72687d8c8f0594c93b3                                                                             0.0s
 => => extracting sha256:01d159b8db2f24da97028c26bf6622e249e162b1adab06a3644c04f1c9fe2dd3                                                                             0.0s
 => => extracting sha256:5fb9a81470f3644c474192baf0827a34749286cb6d933091d4d4463ea4f9c495                                                                             0.0s
 => => extracting sha256:9b1e1e7164db75ad0f64e8deeb33e771d455fa590126b2e16d25e5a75fc6f517                                                                             0.0s
 => => extracting sha256:92c32b2199497c4a79c29180e33dc5fe538334f7dfeb3f916ad14ecb2ea03b6b                                                                             0.0s
 => => extracting sha256:b6f77ae57a55a22b92926d134ef8d86a7234b31859ca866125064294c6c1d576                                                                             0.0s
 => => extracting sha256:965a64b835d2adfac8d1f4f60ca225a822a154552868009a3ef5016218af8d3d                                                                             0.0s
 => => extracting sha256:c0a6b5d6cc3db0fe48b3f0e26bfc28bb98c31fa118db31191d0f78695c42d1c7                                                                             0.0s
 => CACHED apt update                                                                                                                                                 0.0s
 => CACHED apt install busybox-static                                                                                                                                 0.0s
 => CANCELED copy /bin/busybox /bin/busybox                                                                                                                           0.0s
------
 > docker-image://docker.io/library/debian:12-slim:
------
WARN[0013] --debug specified, working folder at /tmp/copa-3853383047 needs to be manually cleaned up 
**Error: failed to solve: failed to load cache key: failed to do request: Head "https://registry-1.docker.io/v2/library/debian/manifests/12-slim": net/http: TLS handshake timeout**

While there is no problem to perform "docker login" to registry-1.docker.io:443 with network and authentication prompt. Please advise.

lioryantov commented 10 months ago

Hi, Any update?

sozercan commented 10 months ago

@lioryantov this is different than your original issue. It seems like a network issue? Is it intermittent? Do you have access to pull from DockerHub?

We have not seen this issue or have any similar reports, can you please provide a repro if you can?

lioryantov commented 10 months ago

@sozercan This is different issue from the original one, and this is not network or permissions issue to access DockerHub registry as mentioned. Any other ideas? Why copa needs to pull images if they already exist on the host and pulled before? can it be configured to resolve locally instead of pulling each time ? Maybe DockerHub closed the connection since there are too much requests sent in same time? Appreciate your help.

sozercan commented 10 months ago

@lioryantov can you run docker pull debian:12-slim and see if it's succesful?

you can also try one of the other methods in https://project-copacetic.github.io/copacetic/website/quick-start#sample-steps instead of running buildkit as a container

If you can provide a sample image or Dockerfile, I can try it on my side.

lioryantov commented 10 months ago

@sozercan I can pull debian:12-slim without problems. Why copa needs to pull images if they already exist on the host and pulled before? Can it be configured to resolve locally instead of pulling each time ? I want to run it when the image located already, no need to pull it each time again and again.

FROM nginx

RUN rm /etc/nginx/conf.d/default.conf
COPY ./ALM/AZ/default.conf /etc/nginx/conf.d/default.conf
RUN rm /etc/nginx/nginx.conf
COPY ./ALM/AZ/nginx.conf /etc/nginx/nginx.conf
COPY ./target/ usr/share/nginx/html/
COPY ./env-config/ usr/share/nginx/html/env-config/

ENV APP_PATH=/opt/app/my-app
WORKDIR ${APP_PATH}

ARG USER_NAME=myuser
ARG GROUP_NAME=mygroup

RUN addgroup --gid 1001 $GROUP_NAME \
    && adduser --system --uid 1000 --gid 1001 --shell /bin/bash --disabled-password $USER_NAME \
    && chown $USER_NAME:$GROUP_NAME ./

RUN chown -R $USER_NAME:$GROUP_NAME ${APP_PATH} && \
        chown -R $USER_NAME:$GROUP_NAME /var/cache/nginx && \
        chown -R $USER_NAME:$GROUP_NAME /var/log/nginx && \
        chown -R $USER_NAME:$GROUP_NAME /usr/share/nginx/html && \
        chown -R $USER_NAME:$GROUP_NAME /etc/nginx/conf.d

RUN touch /var/run/nginx.pid && chown -R $USER_NAME:$GROUP_NAME /var/run/nginx.pid
RUN mkdir ./scripts  && chown -R $USER_NAME:$GROUP_NAME ./scripts

COPY --chown=$USER_NAME:$GROUP_NAME ./ALM/startService.sh  ./scripts/startService.sh
CMD true
COPY --chown=$USER_NAME:$GROUP_NAME ./nextVersion.properties  ./nextVersion.properties
CMD true

RUN chmod -R 777 /etc/nginx/conf.d
RUN chmod -R 777 /usr/share/nginx/html
RUN chmod -R 777 ${APP_PATH}
RUN chmod -R 777 /var
RUN chmod -R 777 /run

USER $USER_NAME
EXPOSE 8080

ENTRYPOINT ${APP_PATH}/scripts/startService.sh

When startService.sh simple script that running few files modifications for nginx configs and contains nginx start command: "nginx -g daemon off; "

sozercan commented 10 months ago

local image resolving is work in progress at #381

sozercan commented 10 months ago

@lioryantov this is just a regular nginx image. rest of the commands are no op as far as copa is concerned. we have this part of our CI https://github.com/project-copacetic/copacetic/blob/main/integration/fixtures/test-images.json#L10C1-L16

I would suggest to try different methods instead of using buildkit as a container

sozercan commented 8 months ago

@lioryantov https://github.com/project-copacetic/copacetic/releases/tag/v0.6.0 is out with local image support. Keep in mind this requires containerd image store.

Closing this issue as complete. Feel free to re-open if you are still seeing issues.

lioryantov commented 8 months ago

@sozercan thank you for the update.

I tried new version of Copa v0.6.0 + Trivy 0.48.1 - still failed to patch. Enabled containerd image snapshotter on my host. Run also docker login to ACR + docker pull of the image myacr.azurecr.io/my-fe-service:1.0.1.1035.

$ docker info -f '{{ .DriverStatus }}'
[[driver-type io.containerd.snapshotter.v1]]
$ docker --version
Docker version 24.0.7, build afdd53b

$ copa patch -i myacr.azurecr.io/my-fe-service:1.0.1.1035 -r my-fe-service.1.0.1.1035.json -t 1.0.1.1035-patched -a tcp://0.0.0.0:$BUILDKIT_PORT --debug --timeout 1m

Output:
DEBU[0000] updates to apply: &{{{debian 12.2} {amd64}} [{curl 7.88.1-10+deb12u4 7.88.1-10+deb12u5 CVE-2023-46218} {curl 7.88.1-10+deb12u4 7.88.1-10+deb12u5 CVE-2023-46219} {libcurl4 7.88.1-10+deb12u4 7.88.1-10+deb12u5 CVE-2023-46218} {libcurl4 7.88.1-10+deb12u4 7.88.1-10+deb12u5 CVE-2023-46219} {libde265-0 1.0.11-1 1.0.11-1+deb12u1 CVE-2023-27103} {libde265-0 1.0.11-1 1.0.11-1+deb12u1 CVE-2023-43887} {libde265-0 1.0.11-1 1.0.11-1+deb12u1 CVE-2023-27102} {libde265-0 1.0.11-1 1.0.11-1+deb12u1 CVE-2023-47471} {libgnutls30 3.7.9-2 3.7.9-2+deb12u1 CVE-2023-5981} {libnghttp2-14 1.52.0-1 1.52.0-1+deb12u1 CVE-2023-44487} {libtiff6 4.5.0-6 4.5.0-6+deb12u1 CVE-2023-3576} {libtiff6 4.5.0-6 4.5.0-6+deb12u1 CVE-2023-40745} {libtiff6 4.5.0-6 4.5.0-6+deb12u1 CVE-2023-41175} {perl-base 5.36.0-7 5.36.0-7+deb12u1 CVE-2023-47038}]} 
DEBU[0000] serving grpc connection                      
[+] Building 29.9s (0/1)                                                                                                                                                          
[+] Building 30.0s (1/1) FINISHED                                                                                                                                                 
 => ERROR resolve image config for myacr.azurecr.io/my-fe-service:1.0.1.1035                                                                                     30.0s
------
 > resolve image config for myac.azurecr.io/my-fe-service:1.0.1.1035:
------

There is a way to tell copa that image is pulled before and exits locally? maybe some flag that I missed (checked help output)

sozercan commented 8 months ago

@lioryantov there's no specific flag for local usage, it should automatically find the image if it exists locally using containerd image store.

Can you paste output of docker buildx ls? Please make sure you are using the default containerd-backed instance (if not, you can set by running docker buildx use default)

lioryantov commented 8 months ago

@sozercan I am using default containerd-backed instance. The issue is still exist

$ docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS  BUILDKIT             PLATFORMS
default * docker                                       
  default default         running v0.11.7+d3e6c1360f6e linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/amd64/v4
$ docker info -f '{{ .DriverStatus }}'
[[driver-type io.containerd.snapshotter.v1]]