search

docker-library / docker

Docker Official Image packaging for Docker
Apache License 2.0
1.14k stars 582 forks source link

docker:24.0.4-git image is broken #434

Closed edvinas31 closed 1 year ago

edvinas31 commented 1 year ago

We are using gitlab ci for docker in docker and when we run this command: git clone https://oauth2:$/{GITLAB_TOKEN}@${CI_SERVER_HOST}/${CSW_GATEWAY_API_PROJECT_PATH} we started to get the error: remote: HTTP Basic: Access denied. The provided password or token is incorrect or your account has 2FA enabled and you must use a personal access token instead of a password. See https://git.tiatechnology.com/help/topics/git/troubleshooting_git#error-on-git-fetch-http-basic-access-denied It only happens in docker:git and docker:24.0.4-git, docker:24.0.5-git, docker:24.0.6-git images (reference https://hub.docker.com/_/docker/tags). So we switched to 24.0.3. After we made a switch we do not get this error anymore

yosifkit commented 1 year ago

There are a few details here that don't match:

If you are just doing a git clone from within the docker:git container, then I am unsure how that could be different, since the git package hasn't changed in Alpine since April: https://git.alpinelinux.org/aports/log/main/git?h=3.18-stable.

$ docker run --rm docker:24.0.3-git apk info -v | grep git
WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.18/main: No such file or directory
WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.18/community: No such file or directory
git-2.40.1-r0
$ docker run --rm docker:24.0.4-git apk info -v | grep git
WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.18/main: No such file or directory
WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.18/community: No such file or directory
git-2.40.1-r0

There is a difference in a few system package versions, but if they are the cause, then there is nothing we can change in the image. It could maybe be one of the following: a deliberate change in Alpine or upstream, or a bug in the Alpine packaging. The difference for the openssl package – openssl, libssl3, and libcrypo3 – is just two CVE patches: https://git.alpinelinux.org/aports/log/main/openssl?h=3.18-stable.

 diff -u <(docker run --rm docker:24.0.3-git apk info -v) <(docker run --rm docker:24.0.4-git apk info -v)
WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.18/main: No such file or directory
WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.18/community: No such file or directory
WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.18/main: No such file or directory
WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.18/community: No such file or directory
--- /dev/fd/63  2023-07-24 10:51:22.519176530 -0700
+++ /dev/fd/62  2023-07-24 10:51:22.519176530 -0700
@@ -22,8 +22,8 @@
 libblkid-2.38.1-r8
 libc-utils-0.7.2-r5
 libcom_err-1.47.0-r2
-libcrypto3-3.1.1-r1
-libcurl-8.1.2-r0
+libcrypto3-3.1.1-r3
+libcurl-8.2.0-r0
 libedit-20221030.3.1-r1
 libexpat-2.5.0-r1
 libidn2-2.3.4-r1
@@ -31,7 +31,7 @@
 libmnl-1.0.5-r1
 libncursesw-6.4_p20230506-r0
 libnftnl-1.2.5-r1
-libssl3-3.1.1-r1
+libssl3-3.1.1-r3
 libtirpc-1.3.3-r2
 libtirpc-conf-1.3.3-r2
 libunistring-1.1-r1
@@ -46,7 +46,7 @@
 openssh-client-common-9.3_p1-r3
 openssh-client-default-9.3_p1-r3
 openssh-keygen-9.3_p1-r3
-openssl-3.1.1-r1
+openssl-3.1.1-r3
 pcre2-10.42-r1
 pigz-2.7-r3
 scanelf-1.3.7-r1
FacioRatio commented 1 year ago

I believe I ran into the same or similar issue.

19 ERROR: failed to prepare vzth5z938h7x0xhvyjugduq3c: invalid argument

[publish 1/1] RUN dotnet publish "Blah.Api.csproj" -c Release -o /app/publish:

WARNING: buildx: git was not found in the system. Current commit information was not captured by the build ERROR: failed to solve: failed to solve with frontend dockerfile.v0: failed to build LLB: failed to prepare vzth5z938h7x0xhvyjugduq3c: invalid argument

edvinas31 commented 1 year ago

There are a few details here that don't match:

  • neither of these images exist yet docker:24.0.5-git and docker:24.0.6-git, did you mean 23.0.5/23.0.6 and maybe 23.* in more of your testing?
  • did you mean this? (note the random extra slash) 
    - https://oauth2:$/{GITLAB_TOKEN}@${CI_SERVER_HOST}/${CSW_GATEWAY_API_PROJECT_PATH}
+ https://oauth2:${GITLAB_TOKEN}@${CI_SERVER_HOST}/${CSW_GATEWAY_API_PROJECT_PATH}

If you are just doing a git clone from within the docker:git container, then I am unsure how that could be different, since the git package hasn't changed in Alpine since April: https://git.alpinelinux.org/aports/log/main/git?h=3.18-stable.

$ docker run --rm docker:24.0.3-git apk info -v | grep git
WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.18/main: No such file or directory
WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.18/community: No such file or directory
git-2.40.1-r0
$ docker run --rm docker:24.0.4-git apk info -v | grep git
WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.18/main: No such file or directory
WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.18/community: No such file or directory
git-2.40.1-r0

There is a difference in a few system package versions, but if they are the cause, then there is nothing we can change in the image. It could maybe be one of the following: a deliberate change in Alpine or upstream, or a bug in the Alpine packaging. The difference for the openssl package – openssl, libssl3, and libcrypo3 – is just two CVE patches: https://git.alpinelinux.org/aports/log/main/openssl?h=3.18-stable.

 diff -u <(docker run --rm docker:24.0.3-git apk info -v) <(docker run --rm docker:24.0.4-git apk info -v)
WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.18/main: No such file or directory
WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.18/community: No such file or directory
WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.18/main: No such file or directory
WARNING: opening from cache https://dl-cdn.alpinelinux.org/alpine/v3.18/community: No such file or directory
--- /dev/fd/63  2023-07-24 10:51:22.519176530 -0700
+++ /dev/fd/62  2023-07-24 10:51:22.519176530 -0700
@@ -22,8 +22,8 @@
 libblkid-2.38.1-r8
 libc-utils-0.7.2-r5
 libcom_err-1.47.0-r2
-libcrypto3-3.1.1-r1
-libcurl-8.1.2-r0
+libcrypto3-3.1.1-r3
+libcurl-8.2.0-r0
 libedit-20221030.3.1-r1
 libexpat-2.5.0-r1
 libidn2-2.3.4-r1
@@ -31,7 +31,7 @@
 libmnl-1.0.5-r1
 libncursesw-6.4_p20230506-r0
 libnftnl-1.2.5-r1
-libssl3-3.1.1-r1
+libssl3-3.1.1-r3
 libtirpc-1.3.3-r2
 libtirpc-conf-1.3.3-r2
 libunistring-1.1-r1
@@ -46,7 +46,7 @@
 openssh-client-common-9.3_p1-r3
 openssh-client-default-9.3_p1-r3
 openssh-keygen-9.3_p1-r3
-openssl-3.1.1-r1
+openssl-3.1.1-r3
 pcre2-10.42-r1
 pigz-2.7-r3
 scanelf-1.3.7-r1

Hi, I see that 24.0.5-git does exist https://hub.docker.com/layers/library/docker/24.0.5-git/images/sha256-ed1845bc9d642f2434db56764b5a6e37b9a893bd92fdf240d1ba089fe261b898?context=explore and when I use that git fails on clone. And regarding second question this is the line which I am using in both versions docker image versions and they fail:

git clone https://oauth2:${GITLAB_TOKEN}@${CI_SERVER_HOST}/${CSW_GATEWAY_API_PROJECT_PATH}

So once again, 24.0.4-git image works for my pipeline and 24.0.5-git does not

yosifkit commented 1 year ago

I am still confused by the inconsistency of exactly which docker images are broken:

Title: docker:24.0.4-git image is broken

It only happens in docker:git and docker:24.0.4-git, docker:24.0.5-git, docker:24.0.6-git images [...]. So we switched to 24.0.3. After we made a switch we do not get this error anymore

So once again, 24.0.4-git image works for my pipeline and 24.0.5-git does not

You keep changing which docker versions work and which don't (with versions that didn't exist) so I can't tell if it might be a change in docker or a change in system packages included or something else entirely (like a server side/proxy change).

I can't reproduce any failure on docker:24.0.5-git; I created a private repo on a hosted gitlab instance and was able to clone using a PAT:

 $ docker pull docker:24.0.5-git
24.0.5-git: Pulling from library/docker
31e352740f53: Already exists
0d7d0e1aee3d: Already exists
616dc7f2241d: Already exists
6b9d31b901ad: Pull complete
f9fa5f10d750: Pull complete
635fc4da2709: Pull complete
60af3bfa01eb: Pull complete
da4338e969cb: Pull complete
bae749acaa72: Pull complete
496be37a29a3: Pull complete
41fdac4f1afc: Pull complete
ce11548e371e: Pull complete
ff0bfb9c0d3f: Pull complete
913d8368b00e: Pull complete
8d6c1d024657: Pull complete
Digest: sha256:d1cb00d2d93f39eaa38aa69f896371096a9b14db5590d338072f8b6b2b18b46a
Status: Downloaded newer image for docker:24.0.5-git
docker.io/library/docker:24.0.5-git
$ docker run -it --rm docker:24.0.5-git sh
/ # GITLAB_TOKEN=[redacted]
/ # git clone "https://oauth2:${GITLAB_TOKEN}@{redacted-server-url}/yosifkit/temp-test.git"
Cloning into 'temp-test'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), done.
/ #

I don't think it is a change in the image, but for every change in the docker images, you can track what changed in the history of this directory of repo-info. This repo scans every official image push and saves the package metadata among other things.

docker:24.0.5* images did not exist when I made the previous comment; they were merged to be built a few hours after my comment in https://github.com/docker-library/official-images/pull/15080.

edvinas31 commented 1 year ago

Good morning, I tried to reproduce this command with docker:git latest docker image and it looks like it started to work again, so probably we will close this issue