search

yannh / kubeconform

A FAST Kubernetes manifests validator, with support for Custom Resources!
Apache License 2.0
2.27k stars 123 forks source link

Better/less verbose error formatting? #261

Closed carlpett closed 8 months ago

carlpett commented 9 months ago

Hey, Since v0.6/#168 the errors on failed validations have gotten a lot more verbose/noisier. I suspect a lot of this comes from the new underlying library, but is there something we can do to support reformatting them?

Example of the same error in v0.5 vs v0.6: ScaledObject so is invalid: For field spec.triggers.0: Additional property metricType is not allowed vs ScaledObject so is invalid: problem validating schema. Check JSON formatting: jsonschema: '/spec/triggers/0' does not validate with file:///var/cache/kubeconform/crds/scaledobject_v1alpha1.json#/properties/spec/properties/triggers/items/additionalProperties: additionalProperties 'metricType' not allowed

We've rolled back after complaints from internal users about it being much harder to figure out what is wrong. For such end users, the schema file location is not very helpful at all, but takes a lot of visual space. Similarly, there's a lot of error wrapping that's relevant when hacking on kubeconform itself.

yannh commented 8 months ago

Hi @carlpett , this is by design - actually made to make it easier to understand exactly what is wrong. In some cases when using registries, seeing what schema the resource is validated against is useful. Apart from the additional schema information, it seems pretty close to what it was before..

There is a JSON output, I guess if you pipe it to jq you could get a custom error message.

$ ./bin/kubeconform -output json fixtures/invalid.yaml { "resources": [ { "filename": "fixtures/invalid.yaml", "kind": "ReplicationController", "name": "bob", "version": "v1", "status": "statusInvalid", "msg": "problem validating schema. Check JSON formatting: jsonschema: '/spec/replicas' does not validate with https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/master-standalone/replicationcontroller-v1.json#/properties/spec/properties/replicas/type: expected integer or null, but got string", "validationErrors": [ { "path": "/spec/replicas", "msg": "expected integer or null, but got string" } ] } ] }

./bin/kubeconform -output json fixtures/invalid.yaml | jq -r '.resources[] | "Error found in "+ .kind + " "+ .name +" in "+ .filename' Error found in ReplicationController bob in fixtures/invalid.yaml

But I don't think I would really remove information from the output as it is now :bow:

carlpett commented 8 months ago

Hi @yannh, Fair enough, perhaps our users or usage (schemas are prepackaged) are atypical. Thanks for the json idea though, I think we can do something with that!