Help validating ArgoCD ApplicationSet - "could not find schema" error

[kevinburkesegment]

I'm trying to validate that the ApplicationSet files I am writing are valid. A CRD exists here, that in theory, I can validate against: https://github.com/argoproj/argo-cd/blob/master/manifests/crds/applicationset-crd.yaml

This gets aggregated into one large swagger.json file here: https://github.com/argoproj/argo-cd/blob/master/assets/swagger.json

I used openapi2jsonschema to convert the swagger.json file into a bunch of subcomponents:

openapi2jsonschema --help ./argoproj/argo-cd/assets/swagger.json

I then uploaded these to a public Github repo. https://github.com/kevinburkesegment/argocd-schemas/

I'm now trying to validate my config file using these. Here is my (abbreviated for space) config file:

myfile.yaml

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: example
  namespace: argocd
spec:
  syncPolicy:
    preserveResourcesOnDeletion: false
  generators:
    - clusters:
        selector:
          matchLabels:
            "segment.com/blah": "true"
    - clusters:
        selector:
          matchExpressions:
            key: "segment.com/blah"
            operator: In
            values:
              - "false"
  template:
    metadata:
      name: "example-{{metadata.labels.clusterNameHash}}"
      labels:
        accountName: "{{metadata.labels.accountName}}"
        kubernetesVersion: "{{metadata.labels.kubernetesVersion}}"
        app: vector
        bundle: monitoring
    spec:
      project: default
      source:
        repoURL: awsaccountid.dkr.ecr.us-west-2.amazonaws.com/charts

I expect this to fail because matchExpressions is supposed to accept a list, not an object.

I'm running e.g. (and to reproduce note you will need to use strict mode)

kubeconform --output json --summary --strict --debug --schema-location 'https://raw.githubusercontent.com/kevinburkesegment/argocd-schemas/master/schemas/{{ .NormalizedKubernetesVersion }}-standalone{{ .StrictSuffix }}/{{ .ResourceAPIVersion }}{{ .ResourceKind }}.json' myfile.yaml

I get the following output:

2022/10/25 14:42:29 using schema found at https://raw.githubusercontent.com/kevinburkesegment/argocd-schemas/master/schemas/master-standalone-strict/v1alpha1applicationset.json
{
  "resources": [
    {
      "filename": "/tmp/canary.yaml",
      "kind": "ApplicationSet",
      "name": "example",
      "version": "argoproj.io/v1alpha1",
      "status": "statusError",
      "msg": "could not find schema for ApplicationSet"
    }
  ],
  "summary": {
    "valid": 0,
    "invalid": 0,
    "errors": 1,
    "skipped": 0
  }
}

I'm confused about this because I see the applicationset is being loaded in the log output. Maybe the tool is not able to find _definitions.json? But I see _definitions.json in https://github.com/instrumenta/kubernetes-json-schema so I know that it must be able to be loaded somehow.

If I try to read the assets/swagger.json file directly, validation succeeds, but "resources" in the output is an empty list, and also, I expect this to fail due to the matchExpressions problem.

Thanks very much for your help, I've been banging my head up against this and appreciate any tips you might have.

[eyarz]

I don't know why you're not able to convert the swagger file into JsonSchmeas that can be parsed by kubeconform, but if you want to validate your ArgoCD resources, you can use kubeconform + CRDs-catalog repo:

kubeconform --output json --summary --schema-location 'https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json' k8s-demo.yaml

This will generate the error that you expected:

{
  "resources": [
    {
      "filename": "k8s-demo.yaml",
      "kind": "ApplicationSet",
      "name": "example",
      "version": "argoproj.io/v1alpha1",
      "status": "statusInvalid",
      "msg": "For field spec.generators.1.clusters.selector.matchExpressions: Invalid type. Expected: array, given: object - For field spec.template.spec: destination is required"
    }
  ],
  "summary": {
    "valid": 0,
    "invalid": 1,
    "errors": 0,
    "skipped": 0
  }
}

[kevinburkesegment]

Thank you, that's helpful!

I don't know why you're not able to convert the swagger file into JsonSchmeas that can be parsed by kubeconform

I'm not sure I follow - I used openapi2jsonschema to convert the swagger.json file into a bunch of subcomponents:

openapi2jsonschema ./argoproj/argo-cd/assets/swagger.json

And then uploaded these to a public Github repo. https://github.com/kevinburkesegment/argocd-schemas/

But maybe you are saying, you're not sure why the tool did not generate complete files.

[eyarz]

But maybe you are saying, you're not sure why the tool did not generate complete files.

yes, I'm not familiar enough with this script to help you debug why the conversion didn't work.

[yannh]

Hi @kevinburkesegment - I'm not too sure either tbh. I would recommend using Datree's CRDs - otherwise, I am considering retiring the Python converter in favor of this other tool https://github.com/yannh/kubeconform/issues/182 maybe you could give this one a shot!