datree test does not catch some invalid indentation

[nosleep77]

Describe the bug I was just testing the datree yaml validation and while testing Probe CRD which we use for monitoring endpoints. It does catch almost all indentation issues but not when I move an attribute back to spec level. 

In my case, I was testing the Probe CRD https://prometheus-operator.dev/docs/kube/blackbox-exporter/ (example at the bottom)

If I move prober, module, targets etc to same level as spec, it does not fail the validation.

However I don't think this is limited to Probe CRD. This issue will exist in other CRs and maybe even native k8s resources but I didn't check yet.

To Reproduce Steps to reproduce the behavior:

1. Have a probe CRD https://prometheus-operator.dev/docs/kube/blackbox-exporter/ (example at the bottom)
2. Move prober section to spec level
3. run datree test on it

Expected behavior I expected it to fail the validation

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: Mac

Datree version (run datree version):

• Version: 1.6.40

Client ID (cat ~/.datree/config.yaml):

• client_id: PTwoXRhdEFzMCZFbtXnkeZ

Additional context
Yaml example can be used at bottom of this page https://prometheus-operator.dev/docs/kube/blackbox-exporter/

However I don't think this is limited to Probe CRD. This issue will exist in other CRs and maybe even native k8s resources but I didn't check yet.

[eyarz]

@nosleep77 Can you apply this "broken" config to your cluster with kubectl? If the answer is no (because the cluster rejects it) - it's a bug in Datree. If the answer is yes - it's not a bug, and I will explain why...

Datree runs three types of validation in the following order: 1) yaml validation 2) schema validation 3) policy check

If I reproduce the steps you mentioned, I can see that it's passing all the checks. Now let's break it down: 1) yaml validation - it's a valid yaml, so this fine 2) schema validation - you're expecting it to fail 3) policy check - passing because you don't have any built-in/custom rule that targets this CR

So the question is - why the schema validation for this CR is passing and not failing? The answer is that the schema we are using to validate this CR is extracted from the Probe CRD that is defined and maintained by the Prometheus community (in this case, Daree is just a pipe to match a file to the correct schema). Therefore, the right place to fix it is by opening a PR/issue in the Prometheus project.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.