search

yannh / kubeconform

A FAST Kubernetes manifests validator, with support for Custom Resources!
Apache License 2.0
2.15k stars 121 forks source link

Performance decrease since 0.6.x #206

Closed jayme-github closed 11 months ago

jayme-github commented 1 year ago

We're using kubeconform v0.4.13 in our CI at https://gerrit.wikimedia.org/g/operations/deployment-charts/ to check a bunch of charts and "deployments". Today I've tried upgrading to v0.6.1 and realized that a complete check of all charts against two k8s versions went from ~20sec to ~70sec (we're running multiple processes in parallel, so those numbers are probably not very representative). As v0.5.0 still has the same performance than v0.4.13 I would suspect https://github.com/yannh/kubeconform/pull/168 to be the culprit.

yannh commented 1 year ago

Hum. 70s sounds like a loooong time - I run this on every commit on quite a few kubeconfigs and it takes a lot less than that... I've not noticed a performance decrease though, esp not in that order of magnitude. Anything special about your use case - very long files, very large amount of files, .. ? I did run some manual performance regression tests when updating the library and couldnt find a decrease in performance :( Maybe I should look at this again.

jayme-github commented 1 year ago

Sorry for getting back so late. I don't think there is anything very special about the single files it's just that we have many of them. We're checking charts (multiple renderings per chart with different values) as well as rendered helmfile templates for all of those charts - all against two different kubernetes versions. I'd say the absolute number is what drives the total runtime so high. If it helps I could maybe find a particular case/input where this is observable.

yannh commented 11 months ago

@jayme-github I'm sorry about this regression for you but I will close this ticket :( I did quite a bit of performance testing when swapping libraries and did end up with really good performance with the new one. If you ever figure out if that's what caused the issue or if it was another commit let me know. Best,

jayme-github commented 11 months ago

Sure, thanks for letting me know. I unfortunately never came around to putting some time into this. I might when we at some point have to move to >= 0.6 :) - cheers