devspace-sh / devspace

DevSpace - The Fastest Developer Tool for Kubernetes ⚡ Automate your deployment workflow with DevSpace and develop software directly inside Kubernetes.
https://devspace.sh
Apache License 2.0
4.19k stars 351 forks source link

Adding docker build config breaks helm chart via illegal labels #2088

Closed gitn00b1337 closed 2 years ago

gitn00b1337 commented 2 years ago

What happened?

When adding the following config the helm deployment breaks for devspace dev:

images:
  svc-test:
    image: ${SVC_TEST_IMAGE}
    dockerfile: ./services/svc-testDockerfile
    context: ./services
    build:
      docker:
        preferMinikube: true
        skipPush: true
        options:
          target: dev
          network: host
          buildArgs:
            NPM_TOKEN: xxxxxxxxx
            PACKAGE_REPO_DOMAIN: "127.0.0.1"

It seems to overwrite the labels in both the kubernetes deployment and service to svc-test-image:dev-Wlt. IE it sets the manifest spec.metadata.labels.{app|name}. I tried getting around this by adding:

deployments:
- name: svc-test
  helm:
    chart:
      name: ./path/to/chart/
    values:
      name: 'svc-test'
      app: 'svc-test'

But this seems to have no effect. I also adjusted the referenced values in the templates from {{ .Values.app }} to {{ .Values.labels.app }} wondering if "app" was reserved but this also changed nothing.

For now I can work around the issue by running devspace build -p development, then devspace dev and having the images snippet above injected through the development profile. It's not ideal though.

Overall, I'd like to say I first tried skaffold and then this. Apart from this issue, devspace is a far better experience so far and I'm loving the simplicity.

What did you expect to happen instead?

I expect it to not place the image name and tag into the selector labels.

How can we reproduce the bug? (as minimally and precisely as possible)

It will take considerable effort to minimise this issue so I will do so after someone has checked my snippets and verified it isn't just something I've misunderstood.

Local Environment:

Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5", GitCommit:"c285e781331a3785a7f436042c65c5641ce8a9e9", GitTreeState:"clean", BuildDate:"2022-03-16T15:51:05Z", GoVersion:"go1.17.8", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.3", GitCommit:"816c97ab8cff8a1c72eccca1026f7820e93e0d25", GitTreeState:"clean", BuildDate:"2022-01-25T21:19:12Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}

/kind bug

gitn00b1337 commented 2 years ago

Forgot to mention, vars is:

vars:
- name: SVC_TEST_IMAGE
  value: svc-test
gitn00b1337 commented 2 years ago

Turns out there isnt a work around. it was using a previous :dev image. Now thats deleted I cant actually use the dynamically generated tag builds with my helm chart. I noticed there's replaceImageTags which can stop this injection behaviour but I can't spot any way to then know what image was built within the yaml :(

gitn00b1337 commented 2 years ago

Had a series of issues. In the end though they can be summed up as:

  1. Devspace was displaying previous error events in the logs. This led to a lot of confusion around which error was actually still relevant.
  2. The right config I needed was:
helm:
    componentChart: false
    replaceImageTags: true
    chart:
      name: ./path/to/chart
    values:
      container:
        image: ${runtime.images.svc-test}:${runtime.images.svc-test.tag}

I didnt see this documented anywhere but was helped on the slack channel.

  1. I also made sure my packages had the org infront. I hadn't bothered previously as its all local-only. Unsure if this had an impact or not.

Thanks for your help on slack! Closing this issue.