yannh / kubeconform

A FAST Kubernetes manifests validator, with support for Custom Resources!
Apache License 2.0
2.15k stars 121 forks source link

Kubeconform does not detect manifests that are too large. #214

Open SleepyBrett opened 1 year ago

SleepyBrett commented 1 year ago

Problem: Given a configmap with a data element that contains a value > 1MiB, kuberenetes will reject the configmap but kubeconform will not flag it as bad.

$ dd if=/dev/urandom bs=786438 count=1 | base64 > /tmp/file
$ kubectl create cm bigandbad --from-file=/tmp/file --dry-run=client -o yaml > bad-cm.yaml
$ kubectl apply -f bad-cm.yaml --dry-run=server
The ConfigMap "bigandbad" is invalid:
* metadata.annotations: Too long: must have at most 262144 bytes
* []: Too long: must have at most 1048576 bytes
$ kubeconform --summary bad-cm.yaml
Summary: 1 resource found in 1 file - Valid: 1, Invalid: 0, Errors: 0, Skipped: 0

Kubeconform should evaluate manifests for size and reject things that exceed limits (I think all manifests are limited to 1mb, but I could be wrong)

eyarz commented 1 year ago

From your description, it sounds like manifest size is a server-side validation, which makes sense because I don't think you can define the file size limit with JsonSchema.

Therefore, this validation is not part of the K8s native objects schemas, and it's not supported by kubeconform but rejected by the cluster (you can read more about it here).

SleepyBrett commented 1 year ago

I get that, and I understand the purity argument of 'we can only do what the schemas tell us.' I would argue that there are also a number of server side validations that apply to ALL manifests, and I think these would be useful and not too complicating.

Off the top of my head:

I agree that testing server side validations for every particular kind is frankly more or less impossible and an endless tarpit. Things like testing that service ports have unique names etc, not to mention kinds outside of the kubernetes core. That said, these 'global' limits I think could be tackled and would make the tool a bit more useful to me at least.

eyarz commented 1 year ago

it's an open-source project, so you are more than welcome to offer those changes to @yannh :)

regarding the other validations, there are 3rd party tools that you can use for that (Datree, Kyverno, Conftest, etc.). I know Datree* has some built-in rules that are doing those server-side validations out of the box, including running kubeconform under the hood.

*discalimer - I'm a maintanier at datree.

yannh commented 1 year ago

Hi @SleepyBrett ! There would definitely be value in being able to reuse directly some Kubernetes server-side validation logic in kubeconform, so we could catch more of these errors. It would be a slight departure from the jsonschema-validation-only approach kubeconform does today but :shrug: maybe it would be a better tool? Happy to discuss approaches for doing so should anyone be willing to submit a patch :bow: