Quite mode to not print Skipped rules

[HariSekhon]

Feature Request to add --quiet mode to not print all the annotation skipped rules as they drown out any unaddressed failures:

+-----------------------------------+-----------------------+
| Enabled rules in policy "Default" | 20                    |
| Configs tested against policy     | 163                   |
| Total rules evaluated             | 2700                  |
| Total rules skipped               | 14                    |
| Total rules failed                | 2                     |
| Total rules passed                | 2684                  |
| See all rules in policy           | https://app.datree.io/ |
+-----------------------------------+-----------------------+

For an example of why this is needed, try finding the 2 failed rules in this output:

https://github.com/HariSekhon/Kubernetes-configs/runs/7407811527?check_suite_focus=true

[HariSekhon]

Also in quiet mode please don't print these redundant messages everywhere:

k8s schema validation skipped: --ignore-missing-schemas flag was used

[adifayer]

@HariSekhon I see your point regarding the "noise" that it can create in the output. As I see it, the skipped rules in the output behave as a "reminder" that some rules that are turned on within the policy are skipped for specific resources, to make sure that this actually expected. I suggest that the --quite flag remove all skipped rules indications, except for the skipped rules count in the aggregated table to keep some kind of indication that there are skipped rules, in cases where it should be revisited. +-----------------------------------+-----------------------+ | Enabled rules in policy "Default" | 20 | | Configs tested against policy | 163 | | Total rules evaluated | 2700 | | Total rules skipped | 14 | | Total rules failed | 2 | | Total rules passed | 2684 | | See all rules in policy | https://app.datree.io/ | +-----------------------------------+-----------------------+ WDYT? :)

[HariSekhon]

Yes I agree, as long as there is a skipped rules counter you will be able to see there are some skipped rules to periodically re-review without them getting in the way of any current issues.

[adifayer]

@HariSekhon Agreed. We will prioritize the issue :)

I would love to suggest a workaround until then- instead of using the skip annotation, you can ignore validating specific resources by creating a custom rule based on the built-in rule, and filter out relevant resources that should be skipped. Attached below is an example of a custom rule I created that verifies readiness probe is configured in all resources except from resources with metadata.name privileged-pod

definitions:
  specContainers:
    if: 
      properties:
        kind:
          enum:
           - Deployment
           - Pod
           - DaemonSet
           - StatefulSet
           - ReplicaSet
           - CronJob
           - Job
        metadata:
          properties:
            name:
              type: string
              not:
                const: privileged-pod    
    then:
      properties:
        spec:
          properties:
            containers:
              items:
                required:
                  - readinessProbe
allOf:
  - $ref: "#/definitions/specContainers"
additionalProperties:
  $ref: "#"
items:
  $ref: "#" 

I checked it on the manifests in your repo, seems like it does the trick 🪄 Here's a link to our docs with further details about custom rules- https://hub.datree.io/custom-rules/custom-rules-overview

[HariSekhon]

I think I'd rather wait for the proper fix than get too deep into customizing the tool only to have to uncustomize it later.

This is a small part of my work after all and it needs to be time efficient.

[adifayer]

Product requirements:

1. When the flag --quiet was passed, excludes the rules skip message from the CLI output (remove the "skipped" section from the output)
2. Keep the skipped rules count in the aggregated summary table