GoogleContainerTools / kaniko

Build Container Images In Kubernetes
Apache License 2.0
14.75k stars 1.43k forks source link

`--registry-mirror` and `--registry-map` do not work when the remapped repository contains dots '.' #3313

Open fbuchmeier-abi opened 1 month ago

fbuchmeier-abi commented 1 month ago

Dear maintainers,

Actual behavior

I have a private mirror of index.docker.io hosted in Harbor. The address of the mirror is harbor.example.com/docker-hub. I can successfully pull images through this mirror with the docker-cli and with containerd (configured in the hosts.toml).

When passing either

--registry-mirror harbor.example.com/hub.docker.com

or

--registry-map=index.docker.io=harbor.example.com/hub.docker.com

to kaniko, kaniko completely ignores the original host (harbor.example.com) and tries to fetch images (here: convco/convco:0.5.0) from https://hub.docker.com instead.

args=(
    "--context=/work/docker"
    "--destination=myUser/helm:latest"
    "--registry-mirror=harbor.example.com/hub.docker.com"
    "--skip-default-registry-fallback"
    "--verbosity=trace"
)
docker run --rm -it -v $PWD:/work -v $HOME/.docker:/.docker -e DOCKER_CONFIG=/.docker gcr.io/kaniko-project/executor:debug ${args[@]}
DEBU[0000] registry-map remaps index.docker.io to harbor.example.com/hub.docker.com. 
DEBU[0000] Copying file /work/docker/Dockerfile to /kaniko/Dockerfile 
TRAC[0000] Adding /var/run to default ignore list       
DEBU[0000] Skip resolving path /kaniko/Dockerfile       
DEBU[0000] Skip resolving path /work/docker             
DEBU[0000] Skip resolving path /cache                   
DEBU[0000] Skip resolving path                          
DEBU[0000] Skip resolving path                          
DEBU[0000] Skip resolving path                          
DEBU[0000] Skip resolving path                          
DEBU[0000] Built stage name to index map: map[]         
DEBU[0000] Found extra base image stage convco/convco:0.5.0 
INFO[0000] Retrieving image manifest convco/convco:0.5.0 
INFO[0000] Retrieving image convco/convco:0.5.0 from mapped registry harbor.example.com 
WARN[0001] Failed to retrieve image convco/convco:0.5.0 from remapped registry harbor.example.com: unable to complete operation after 0 attempts, last error: GET https://hub.docker.com/v2/: unexpected status code 404 Not Found: <!doctype html>
<html lang="en">

<head>
    <meta charset="utf-8">
    <title>404 : Docker</title>

This only happens when the context path for the new repository contains dots ., so hub-docker-com works fine and is correctly resolved to harbor.example.com/hub-docker-com but hub.docker.com does not.

This is also evident in the unit tests (remote_test.go) on the current head e328007bc1fa0d8c2eacf1918bebbabc923abafa:

diff --git a/pkg/image/remote/remote_test.go b/pkg/image/remote/remote_test.go
index c1d2300f..ddccf311 100644
--- a/pkg/image/remote/remote_test.go
+++ b/pkg/image/remote/remote_test.go
@@ -152,6 +152,20 @@ func Test_remapRepository(t *testing.T) {
                        newRepositoryPrefix: "subdir1/subdir2/",
                        expectedRepository:  "newreg.io/subdir1/subdir2/namespace/debian",
                },
+               {
+                       name:                "Test case 10",
+                       repository:          "debian",
+                       newRegistry:         "newreg.io",
+                       newRepositoryPrefix: "index.docker.io/",
+                       expectedRepository:  "newreg.io/index.docker.io/library/debian",
+               },
+               {
+                       name:                "Test case 11",
+                       repository:          "debian",
+                       newRegistry:         "newreg.io",
+                       newRepositoryPrefix: "index-docker-io/",
+                       expectedRepository:  "newreg.io/index-docker-io/library/debian",
+               },
                // Add more test cases here
        }
=== RUN   Test_remapRepository
=== RUN   Test_remapRepository/Test_case_1
=== RUN   Test_remapRepository/Test_case_2
=== RUN   Test_remapRepository/Test_case_3
=== RUN   Test_remapRepository/Test_case_4
=== RUN   Test_remapRepository/Test_case_5
=== RUN   Test_remapRepository/Test_case_6
=== RUN   Test_remapRepository/Test_case_7
=== RUN   Test_remapRepository/Test_case_8
=== RUN   Test_remapRepository/Test_case_9
=== RUN   Test_remapRepository/Test_case_10
    remote_test.go:184: index.docker.io/library/debian should have been normalized to newreg.io/index.docker.io/library/debian, got index.docker.io/library/debian
=== RUN   Test_remapRepository/Test_case_11
--- FAIL: Test_remapRepository (0.00s)
    --- PASS: Test_remapRepository/Test_case_1 (0.00s)
    --- PASS: Test_remapRepository/Test_case_2 (0.00s)
    --- PASS: Test_remapRepository/Test_case_3 (0.00s)
    --- PASS: Test_remapRepository/Test_case_4 (0.00s)
    --- PASS: Test_remapRepository/Test_case_5 (0.00s)
    --- PASS: Test_remapRepository/Test_case_6 (0.00s)
    --- PASS: Test_remapRepository/Test_case_7 (0.00s)
    --- PASS: Test_remapRepository/Test_case_8 (0.00s)
    --- PASS: Test_remapRepository/Test_case_9 (0.00s)
    --- FAIL: Test_remapRepository/Test_case_10 (0.00s)
    --- PASS: Test_remapRepository/Test_case_11 (0.00s)

Expected behavior

Registry Mirrors with dots . in their repository names get resolved correctly.

Additional Information

Dockerfile

ARG ALPINE_VERSION=3.20

FROM alpine:${ALPINE_VERSION}

COPY --from=convco/convco:0.5.0 /entrypoint /usr/local/bin/convco
RUN convco --version

Kaniko Image - gcr.io/kaniko-project/executor@sha256:c3109d5926a997b100c4343944e06c6b30a6804b2f9abe0994d3de6ef92b028e

I will did into the code and see if this can be fixed quickly but just in case this rings a bell, any help is appreciated :)

Best regards, Florian.

Triage Notes for the Maintainers

Description Yes/No
Please check if this a new feature you are proposing
  • - [ ]
Please check if the build works in docker but not in kaniko
  • - [ ]
Please check if this error is seen when you use --cache flag
  • - [ ]
Please check if your dockerfile is a multistage dockerfile
  • - [ ]
next-jesusmanuelnavarro commented 2 days ago

I think this is a duplicate of #3182.