Feature request: Delete pre `1.11` (Cluster)PolicyReports when upgrading to `1.11`

[viccuad]

Pre 1.11, (Cluster)PolicyReports where created as follows:

• 1 ClusterPolicyReport named polr-clusterwide
• 1 PolicyReport per namespace, named polr-ns-<namespace name>.

From 1.11 onwards, (Cluster)PolicyReports are created as follows:

• Per each resource, create a (Cluster)PolicyReport with metadata.name that equals the resource uid.

The audit-scanner in 1.11 doesn't deal with the old policy reports format, leaving them hanging behind. Still, downstream consumers such as Rancher Kubewarden UI and Policy Reporter UI would read both the old format and the new, potentially showing old report results that are bound to be outdated.

Acceptance criteria

• [x] Add a Helm pre-install update hook that runs the kubewarden/kubectl image and deletes the old reports created by kubewarden. The job should not delete all kubewarden reports (pre and post 1.11); as this means that an upgrade cleans the reports, which is too invasive: https://github.com/kubewarden/helm-charts/pull/410
• [x] Update docs https://github.com/kubewarden/docs/pull/379
• [x] Update rfc https://github.com/kubewarden/rfc/pull/32

[viccuad]

I have the Helm template, and I'm iterating with the kubectl delete command. Note that the kubewarden/kubectl image has as entrypoint kubectl and doesn't start a shell, so we can't do fancy things.

The following command deletes all reports owned by Kubewarden, pre and post 1.11. I find this too invasive since this will happen in all helm upgrade:

kubectl delete clusterpolicyreport,policyreport \
  -l app.kubernetes.io/managed-by=kubewarden

The following command tries to delete those reports that match the pre 1.11 metadata.name schema, but this is not possible, as neither field selectors nor set-based selectors allow for wildcards or prefixes:

kubectl delete clusterpolicyreport,policyreport \
  -l app.kubernetes.io/managed-by=kubewarden \
  --field-selector metadata.name==polr-clusterwide \ # correct
  --field-selector metadata.name==polr-ns-<ns name> # not doable

Hence, I would like to add a new label to 1.11 reports that would allow to target the pre 1.11 reports, because they would be missing this new label as only 1.11 reports will have it:

metadata:
  labels:
    app.kubernetes.io/managed-by: kubewarden
    kubewarden.io/policyreport-version: v2

[flavio]

I think this is a good idea. In that way the delete could use the selector kubewarden.io/policyreport-version not in "v2", or something like that

[viccuad]

the selector should be matching for the label to not be present, or we may delete v3 policyreports in the future for example. The ideal would have been to match v1 for deletion, but, well, hindsight 50/50.

[viccuad]

There doesn't seem to be a way to match for missing labels, so the job will grow in time as follows:

kubectl delete clusterpolicyreport,policyreport \
  -l app.kubernetes.io/managed-by=kubewarden \
  --field-selector kubewarden.io/policyreport-version!=v2 \
  --field-selector kubewarden.io/policyreport-version!=v3 \
  etc

But by that time, we can assume v1 reports are not around, and match for ==v3 or ==v4 anyways.

Edit: This suffices,

kubectl get clusterpolicyreport,policyreport \
  -l app.kubernetes.io/managed-by=kubewarden \
  -l '!kubewarden.io/policyreport-version'