VictoriaMetrics / operator

Kubernetes operator for Victoria Metrics
Apache License 2.0
430 stars 142 forks source link

can't install victoria-metrics-k8s-stack with argocd appset and remotewrite tlsconfig set #1132

Closed fredleger closed 1 day ago

fredleger commented 1 day ago

application set manifest

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: victoria-metrics
  namespace: argocd
  annotations:
    argocd.argoproj.io/sync-wave: '0'
spec:
  generators:
    - list:
        elements:
          - cluster: dev
            url: https://somecluster
  template:
    metadata:
      name: victoria-metrics-{{cluster}}
      namespace: argocd
      annotations:
        argocd.argoproj.io/sync-wave: '1'
      labels:
        level: infrastructure
        tier: victoria-metrics
      finalizers:
        - resources-finalizer.argocd.argoproj.io
    spec:
      project: default
      destination:
        server: '{{url}}'
        namespace: monitoring
      syncPolicy:
        # TODO: uncomment when fully tested
        # automated:
        #   prune: false
        #   selfHeal: false
        syncOptions:
          - CreateNamespace=true
          - ServerSideApply=true
          - SkipDryRunOnMissingResource=true
          - Validate=false
          # https://argo-cd.readthedocs.io/en/stable/user-guide/sync-options/#respect-ignore-difference-configs
          # argocd must also ignore difference during apply stage
          # otherwise it ll silently override changes and cause a problem
          - RespectIgnoreDifferences=true
      ignoreDifferences:
        - group: ''
          kind: Secret
          name: victoria-metrics-{{cluster}}-victoria-metrics-operator-validation
          namespace: monitoring
          jsonPointers:
            - /data
        - group: admissionregistration.k8s.io
          kind: ValidatingWebhookConfiguration
          name: victoria-metrics-{{cluster}}-victoria-metrics-operator-admission
          jqPathExpressions:
            - .webhooks[]?.clientConfig.caBundle
      sources:
        - repoURL: https://victoriametrics.github.io/helm-charts/
          chart: victoria-metrics-k8s-stack
          targetRevision: 0.27.*
          helm:
            valueFiles:
              - $values/values/vm-agent.yaml
            # appsets templating is not supported in external helm values file
            values: |
              global:
                clusterLabel: '{{cluster}}'
              vmagent:
                spec:
                  externalLabels:
                    cluster: '{{cluster}}'
        - repoURL: git@gitlab.com:somerepo.git
          targetRevision: HEAD
          ref: values

values

...
vmagent:
  enabled: true
  spec:
    externalLabels:
      cluster: '{{cluster}}'
    secrets:
      - vmagent-tls
    remoteWrite:
      - url: https://vminsert.someurl/insert/0/prometheus/api/v1/write
        tlsConfig:
          # note that this secrets are created by extraObjects at the end of the values file
          caFile: /etc/vm/secrets/vmagent-tls/ca.crt
          certFile: /etc/vm/secrets/vmagent-tls/client.crt
          keyFile: /etc/vm/secrets/vmagent-tls/client.key
          verify: false
...

error

failed to create typed patch object (monitoring/vm; operator.victoriametrics.com/v1beta1, Kind=VMAgent): .spec.remoteWrite[0].tlsConfig.verify: field not declared in schema

The same config is working on pure helm install

fredleger commented 1 day ago

similar to https://github.com/VictoriaMetrics/operator/issues/215 ?

fredleger commented 1 day ago

got it it's insecureSkipVerify and not verify. Sorry for the noise.