grafana / k6-operator

An operator for running distributed k6 tests.
Apache License 2.0
593 stars 166 forks source link

When specifying values in the Helm chart, the installation fails. #471

Closed YusukeSugiyamaTG closed 3 weeks ago

YusukeSugiyamaTG commented 1 month ago

Brief summary

When using Argo CD and Helm chart for installation, specifying tolerations and affinity as follows will cause an error.

repoURL: https://grafana.github.io/helm-charts
chart: k6-operator
targetRevision: 3.9.0
helm:
  releaseName: k6-operator
  valuesObject:
    tolerations:
      - key: "dedicated"
        operator: "Equal"
        value: "platform"
        effect: "NoSchedule"
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
                - key: service-name
                  operator: In
                  values:
                    - platform

Error details

Error: values don't meet the specifications of the schema(s) in the following chart(s): k6-operator: - tolerations: Invalid type. Expected: object, given: array - affinity: Additional property nodeAffinity is not allowed - affinity: Additional property podAntiAffinity is not allowed

I think that values.schema.json was added with this upgrade.

https://github.com/grafana/k6-operator/commit/735f8759b2030c3441e61ab8477bb1d03e5555c4#diff-5b1691e101964ac9ed5929dabe795fa6d9a722e062c88d3245e23a06b76a6317R5

    "affinity": {
      "additionalProperties": false,
      "description": "affinity -- Affinity to be applied on all containers",
      "title": "affinity",
      "type": "object"
    },

    "tolerations": {
      "additionalProperties": false,
      "description": "tolerations -- Tolerations to be applied on all containers",
      "title": "tolerations",
      "type": "object"
    }

Here, the additionalProperties of affinity is set to false, so it seems that additional elements cannot be specified in affinity. Regarding tolerations, it seems to be a pure type error.

k6-operator version or image

0.0.17

Helm chart version (if applicable)

3.9.0

TestRun / PrivateLoadZone YAML

This does not apply in this case.

Other environment details (if applicable)

No response

Steps to reproduce the problem

The following definition and execution in the application.yaml file of Argo CD will cause an error.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  finalizers:
    - resources-finalizer.argocd.argoproj.io
  name: k6-operator
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://grafana.github.io/helm-charts
    chart: k6-operator
    targetRevision: 3.9.0
    helm:
      releaseName: k6-operator
      valuesObject:
        tolerations:
          - key: "dedicated"
            operator: "Equal"
            value: "platform"
            effect: "NoSchedule"
        affinity:
          nodeAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              nodeSelectorTerms:
                - matchExpressions:
                    - key: service-name
                      operator: In
                      values:
                        - platform
          podAntiAffinity:
            preferredDuringSchedulingIgnoredDuringExecution:
              - weight: 100
                podAffinityTerm:
                  labelSelector:
                    matchExpressions:
                      - key: app.kubernetes.io/name
                        operator: In
                        values:
                          - k6-operator
                  topologyKey: topology.kubernetes.io/zone
  destination:
    namespace: k6-operator-system
    server: https://kubernetes.default.svc

Expected behaviour

Using the application.yaml mentioned above, I expect that the installation will be completed successfully with Argo CD and Helm chart.

Actual behaviour

As mentioned above, the following error occurs.

Error details

Error: values don't meet the specifications of the schema(s) in the following chart(s): k6-operator: - tolerations: Invalid type. Expected: object, given: array - affinity: Additional property nodeAffinity is not allowed - affinity: Additional property podAntiAffinity is not allowed

yorugac commented 1 month ago

Hi @YusukeSugiyamaTG, yes, it seems the schema file is incorrect after all 😞 It was added on the request for the specific use case and locally it seemed to work, AFAIR. Thanks for the report!

I'll need to look into this, perhaps even remove the schema file completely if it causes such problems. Meanwhile, I believe you can just switch off the schema step with --skip-schema-validation.

yorugac commented 1 month ago

The quickest way to repeat this (without ArgoCD): install Helm as per instructions, providing values.yaml with affinity and tolerations, etc.

YusukeSugiyamaTG commented 1 month ago

Hi @yorugac , thank you very much for your response. As you suggested, I will try using the --skip-schema-validation option and following the instructions. I would be very happy if it worked on Argo CD, so I would be grateful if you could consider it.

yorugac commented 3 weeks ago

Re-opening for now: most of the fixes were done in https://github.com/grafana/k6-operator/pull/474 But there is still a fix for tolerations left, a WIP in https://github.com/grafana/k6-operator/pull/481

yorugac commented 3 weeks ago

Fix for tolerations was merged. And a new Helm release was just made: https://github.com/grafana/helm-charts/releases/tag/helm-k6-operator-3.9.1 With that, closing this issue 🤞

YusukeSugiyamaTG commented 3 weeks ago

@yorugac Thank you!! 😄