search

yannh / kubeconform

A FAST Kubernetes manifests validator, with support for Custom Resources!
Apache License 2.0
2.15k stars 121 forks source link

Regression on 0.6.3: could not find schema due to Draft 4 being used #223

Open carlossg opened 1 year ago

carlossg commented 1 year ago

Hi, trying 0.6.3 I noticed a regression, caused by #221

with 0.6.3 it breaks

❯ ~/Downloads/kubeconform-063/kubeconform -debug -strict -schema-location target/kubernetes-json-schema/master target/test.yaml
2023/07/18 20:04:30 using schema found at target/kubernetes-json-schema/master/master-standalone-strict/verticalpodautoscaler-autoscaling-v1beta2.json
target/test.yaml - VerticalPodAutoscaler toggleservice-vpa failed validation: could not find schema for VerticalPodAutoscaler

with 0.6.2 it worked fine

❯ ~/Downloads/kubeconform-062/kubeconform -debug -strict -schema-location target/kubernetes-json-schema/master target/test.yaml
2023/07/18 20:04:34 using schema found at target/kubernetes-json-schema/master/master-standalone-strict/verticalpodautoscaler-autoscaling-v1beta2.json

what happens is that the schema is not valid in draft 4 but it is in the default 2020 schema

2023/07/18 20:28:18 error jsonschema file:///.../target/kubernetes-json-schema/master/master-standalone-strict/verticalpodautoscaler-autoscaling-v1beta2.json compilation failed: '/properties/spec/required' does not validate with http://json-schema.org/draft-04/schema#/properties/properties/additionalProperties/$ref/properties/required/$ref/minItems: minimum 1 items required, but found 0 items

from https://github.com/yannh/kubeconform/issues/217#issuecomment-1635675208 I assume we need to fix our schemas to be Draft 4 ?

carlossg commented 1 year ago

if that's the case it would be good to push this to a 0.7.0 version at least and have a changelog warning

yannh commented 1 year ago

i guess it should use the draft version specified in the schema and only fallback to draft 4 🤔

carlossg commented 1 year ago

I've tried with some public schemas and it works, so I'll change ours. It would be good to have more debug entries inside downloadSchema. I see that the debug option is not available there to conditionally print logs

❯ ~/dev/yannh/kubeconform/bin/kubeconform -debug -strict -schema-location 'https://raw.githubusercontent.com/zapier/kubechecks/main/schemas/v1.23.0/{{ .ResourceKind }}{{ .KindSuffix }}.json' target/test.yaml
2023/07/18 20:44:08 using schema found at https://raw.githubusercontent.com/zapier/kubechecks/main/schemas/v1.23.0/verticalpodautoscaler-autoscaling-v1beta2.json
carlossg commented 1 year ago

@yannh I cannot find many sources of autoscaler or its swagger definition, do you know where to get them? maybe to contribute them to https://github.com/yannh/kubernetes-json-schema/

yannh commented 11 months ago

I agree it would be good to have a way to have debug info in downloadschema, but I ve found it can be really verbose since well, by architecture, when you use multiple schema repositories it will try to find every file in every repository until it finds it...

yannh commented 11 months ago

@carlossg any chance you could share the schema so I could reproduce the original issue? I feel like Kubeconform should complain if it downloads an invalid schema!

carlossg commented 10 months ago

it was this one I think

{
  "type": "object",
  "properties": {
    "spec": {
      "type": "object",
      "required": [],
      "properties": {
        "targetRef": {
          "type": "object"
        },
        "updatePolicy": {
          "type": "object",
          "properties": {
            "minReplicas": {
              "type": "integer"
            },
            "updateMode": {
              "type": "string"
            }
          },
          "additionalProperties": false
        },
        "resourcePolicy": {
          "type": "object",
          "properties": {
            "containerPolicies": {
              "type": "array",
              "items": {
                "type": "object",
                "properties": {
                  "containerName": {
                    "type": "string"
                  },
                  "controlledValues": {
                    "type": "string",
                    "enum": [
                      "RequestsAndLimits",
                      "RequestsOnly"
                    ]
                  },
                  "mode": {
                    "type": "string",
                    "enum": [
                      "Auto",
                      "Off"
                    ]
                  },
                  "minAllowed": {
                    "type": "object"
                  },
                  "maxAllowed": {
                    "type": "object"
                  },
                  "controlledResources": {
                    "type": "array",
                    "items": {
                      "type": "string",
                      "enum": [
                        "cpu",
                        "memory"
                      ]
                    }
                  }
                },
                "additionalProperties": false
              }
            }
          },
          "additionalProperties": false
        }
      },
      "additionalProperties": false
    }
  }
}
rino-pupkin commented 10 months ago

Can confirm the issue. Looks like it found a correct schema but throws error:

root@gitlab-runner-vm:/home/gitlab-runner# cat tmp.yaml | kubeconform -kubernetes-version 1.28.1 -schema-location default -schema-location '/home/gitlab-runner/yamltests/schemas/{{ .ResourceKind }}_{{ .ResourceAPIVersion }}.json' -n 1 -summary -strict -skip Certificate  --debug
2023/10/31 14:32:48 using schema found at https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/v1.28.1-standalone-strict/configmap-v1.json
2023/10/31 14:32:48 using schema found at https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/v1.28.1-standalone-strict/service-v1.json
2023/10/31 14:32:48 using schema found at https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/v1.28.1-standalone-strict/deployment-apps-v1.json
2023/10/31 14:32:48 using schema found at https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/v1.28.1-standalone-strict/ingress-networking-v1.json
2023/10/31 14:32:48 could not find schema at https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/v1.28.1-standalone-strict/servicemonitor-monitoring-v1.json
2023/10/31 14:32:48 using schema found at /home/gitlab-runner/yamltests/schemas/servicemonitor_v1.json
2023/10/31 14:32:48 could not find schema at https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/v1.28.1-standalone-strict/verticalpodautoscaler-autoscaling-v1.json
2023/10/31 14:32:48 using schema found at /home/gitlab-runner/yamltests/schemas/verticalpodautoscaler_v1.json
stdin - VerticalPodAutoscaler xxx failed validation: could not find schema for VerticalPodAutoscaler
2023/10/31 14:32:48 using schema found at https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/v1.28.1-standalone-strict/job-batch-v1.json
Summary: 11 resources found parsing stdin - Valid: 9, Invalid: 0, Errors: 1, Skipped: 1

root@gitlab-runner-vm:/home/gitlab-runner# echo $?
1

v0.6.2 works great we use this schema to validate vpa https://raw.githubusercontent.com/kubernetes/autoscaler/master/vertical-pod-autoscaler/deploy/vpa-v1-crd.yaml

and this to validate servicemonitor https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/release-0.60/example/prometheus-operator-crd-full/monitoring.coreos.com_servicemonitors.yaml

we use openapi2jsonschema.py to downlad and convert schemas into json.

dctrwatson commented 10 months ago

Here's the VPA CRD that will validate with kubeconform v0.6.3+ : https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/deploy/vpa-v1-crd-gen.yaml

yannh commented 9 months ago

:wave: My understanding is that the schemas should be in draft 4... Am I mistaken? See https://github.com/yannh/kubeconform/issues/217#issuecomment-1635675208 - I would close this ticket unless a lot of public CRDs aren't in that format?

megamorf commented 2 months ago

Hey guys, I'd like to report something that is somewhat related. It might help other users who struggled with this like we did.

Starting with v0.6.3 we got the "could not find schema" for an RDSInstance resource:

RDSInstance dcpc-commerce-markets failed validation: could not find schema for RDSInstance

I checked with:

In our case this is a crossplane resource and the CRD is managed by a different team. I converted the CRD spect to json-schema format and added the validation fields:

  {
+   "id": "http://json-schema.org/draft-04/schema#",
+   "$schema": "http://json-schema.org/draft-04/schema#",
    "properties": {
      "apiVersion": {
        "type": "string"
      },
      "kind": {
        "type": "string"
      },
  [...]

As it turns out, there was an enum list with a duplicate in it:

image

As soon as the duplicate was removed the could not find schema error disappeared and the validation started working on all recent versions:

Before

KUBECONFORM [0.6.2]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets is valid
Summary: 3 resources found in 1 file - Valid: 3, Invalid: 0, Errors: 0, Skipped: 0

KUBECONFORM [0.6.3]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets failed validation: could not find schema for RDSInstance
Summary: 3 resources found in 1 file - Valid: 2, Invalid: 0, Errors: 1, Skipped: 0

KUBECONFORM [0.6.4]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets failed validation: could not find schema for RDSInstance
Summary: 3 resources found in 1 file - Valid: 2, Invalid: 0, Errors: 1, Skipped: 0

KUBECONFORM [0.6.5]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets failed validation: could not find schema for RDSInstance
Summary: 3 resources found in 1 file - Valid: 2, Invalid: 0, Errors: 1, Skipped: 0

KUBECONFORM [0.6.6]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets failed validation: could not find schema for RDSInstance
Summary: 3 resources found in 1 file - Valid: 2, Invalid: 0, Errors: 1, Skipped: 0

After

KUBECONFORM [0.6.2]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets is valid
Summary: 3 resources found in 1 file - Valid: 3, Invalid: 0, Errors: 0, Skipped: 0

KUBECONFORM [0.6.3]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets is valid
Summary: 3 resources found in 1 file - Valid: 3, Invalid: 0, Errors: 0, Skipped: 0

KUBECONFORM [0.6.4]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets is valid
Summary: 3 resources found in 1 file - Valid: 3, Invalid: 0, Errors: 0, Skipped: 0

KUBECONFORM [0.6.5]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets is valid
Summary: 3 resources found in 1 file - Valid: 3, Invalid: 0, Errors: 0, Skipped: 0

KUBECONFORM [0.6.6]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets is valid
Summary: 3 resources found in 1 file - Valid: 3, Invalid: 0, Errors: 0, Skipped: 0