localRegistry being used even though my cluster registries work fine

[igoooor]

What happened? I have a local cluster created with k3d and I used the option --registry-create myregistry.localhost to have its own registry which I can push from my local machine and that the cluster can use to pull container images. Even though docker push works fine from my local machine, when I start my devspace project I receive this output:

Ensuring image pull secret for registry: myregistry.localhost:5000...
Couldn't retrieve username for registry myregistry.localhost:5000 from docker store
Couldn't retrieve password for registry myregistry.localhost:5000 from docker store
local-registry: Starting Local Image Registry
local-registry: Port forwarding to local registry started on: 30521 -> 5000
Ensuring image pull secret for registry: myregistry.localhost:5000...
Couldn't retrieve username for registry myregistry.localhost:5000 from docker store
Couldn't retrieve password for registry myregistry.localhost:5000 from docker store
Ensuring image pull secret for registry: myregistry.localhost:5000...

But as I said, I can docker push from the same machine/environment where I start my devspace project, so I don't understand why devspace says that. Plus, if in my devspace.yaml file I add:

localRegistry:
  enabled: false

it will still display the Couldn't retrieve... message, but it will actually correctly push to that registry.

What did you expect to happen instead? I expected to not see these errors, and that it does not fallback to the localRegistry

How can we reproduce the bug? (as minimally and precisely as possible)

My devspace.yaml:

version: v2beta1
images:
  php:
    image: myregistry.localhost:5000/platform/php
    tags:
      - latest
    dockerfile: ../../.ops/env/base/php/docker/Dockerfile
    context: ../../.ops/env/base/php/docker/
    buildArgs:
      APP_ENV: dev
      APP_DEBUG: "true"
      BUILD_ENV: local
      POST_INSTALL_HARDENING: "false"
      USER_ID: $(id -u)
  nginx:
    image: myregistry.localhost:5000/platform/nginx
    tags:
      - latest
    dockerfile: ../../.ops/env/base/nginx/docker/Dockerfile
    context: ../../.ops/env/base/nginx/docker/
  node:
    image: myregistry.localhost:5000/platform/node
    tags:
      - latest
    dockerfile: ../../.ops/env/global/node/docker/Dockerfile
    context: ../../app/metrics

Local Environment:

• DevSpace Version: 6.2.3
• Operating System: mac m1
• ARCH of the OS: ARM64 Kubernetes Cluster:
• Local: k3d
• Kubernetes Version:

Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.3", GitCommit:"434bfd82814af038ad94d62ebe59b133fcb50506", GitTreeState:"clean", BuildDate:"2022-10-12T10:47:25Z", GoVersion:"go1.19.2", Compiler:"gc", Platform:"darwin/arm64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.6+k3s1", GitCommit:"418c3fa858b69b12b9cefbcff0526f666a6236b9", GitTreeState:"clean", BuildDate:"2022-04-28T22:16:58Z", GoVersion:"go1.17.5", Compiler:"gc", Platform:"linux/arm64"}

Anything else we need to know?

[FabianKramm]

Hey @igoooor ! Thanks for creating this issue! Would it be possible to share your images section so that we know what builder etc you are using?

[igoooor]

thanks @FabianKramm I updated my initial message

[FabianKramm]

Hey @igoooor! Your problem should be fixed with https://github.com/loft-sh/devspace/pull/2486, but it seems the error message comes up because DevSpace cannot find the credentials locally as this seems to be an insecure registry. So when disabling the localRegistry, is DevSpace correctly pushing to the registry?

[igoooor]

Yes, when disabling the localRegistry, the error still shows up but DevSpace is correctly pushing to the registry. And you are also correct this is an insecure registry, so there is no credentials to be found.

[igoooor]

I also should specify that it not only happens to me on my ARM64 Macbook M1, but also to colleagues with Fedora amd64 machines.

[FabianKramm]

With DevSpace v6.2.4 the localRegistry configuration shouldn't be needed anymore and if you set createPullSecret: false for the images the warning will also not appear anymore

[igoooor]

many thanks @FabianKramm 🙌 I will test that new release today

[igoooor]

I think I must be hitting some edge case here with my macOS. The yaml I gave you is the one linux users are using (I'm the only mac user in the team). macOS (by default at least) does not support xxx.localhost (it won't resolve to localhost), so I am using instead myregistry.local.gd instead. local.gd is a service similar to nip.ip xip.io and others, where all subdomains will resolve to localhost. Do you see something I could do? Because now with createPullSecret: false and localRegistry: enabled: false it gives me this output:

info Using kube context 'k3d-localcluster'
build:nginx Rebuild image registry.local.gd:22345/platform/nginx because image config has changed
build:nginx Building image 'localhost:30944/platform/nginx:latest' with engine 'docker'
build:node Rebuild image registry.local.gd:22345/platform/node because image config has changed
build:nginx Authenticating (localhost:30944)...
build:node Building image 'localhost:30944/platform/node:latest' with engine 'docker'
build:php Rebuild image registry.local.gd:22345/platform/php because image config has changed
build:node Authenticating (localhost:30944)...
build:php Building image 'localhost:30944/platform/php:latest' with engine 'docker'
build:php Authenticating (localhost:30944)...
build_images: build images: error building image registry.local.gd:22345/platform/php:latest: Error during image registry authentication: Error response from daemon: Get "http://localhost:30944/v2/": dial tcp [::1]:30944: connect: connection refused

[igoooor]

If I only set localRegistry: enabled: false it will work with the following output:

info Using kube context 'k3d-localcluster'
Ensuring image pull secret for registry: registry.local.gd:22345...
Ensuring image pull secret for registry: registry.local.gd:22345...
Ensuring image pull secret for registry: registry.local.gd:22345...
build:nginx Rebuild image registry.local.gd:22345/platform/nginx because tag is missing
build:nginx Building image 'registry.local.gd:22345/platform/nginx:latest' with engine 'docker'
build:node Rebuild image registry.local.gd:22345/platform/node because tag is missing
build:nginx Authenticating (registry.local.gd:22345)...
build:node Building image 'registry.local.gd:22345/platform/node:latest' with engine 'docker'
build:php Rebuild image registry.local.gd:22345/platform/php because tag is missing
build:node Authenticating (registry.local.gd:22345)...
build:php Building image 'registry.local.gd:22345/platform/php:latest' with engine 'docker'
build:php Authenticating (registry.local.gd:22345)...
build:nginx Authentication successful (registry.local.gd:22345)
build:php Authentication successful (registry.local.gd:22345)
build:node Authentication successful (registry.local.gd:22345)

I guess this is totally acceptable right?