[RFE] Backup Operator ignores "*.*" in EncryptionConfiguration

[bvankampen]

Rancher Server Setup

• Rancher version: 2.9.2
• Backup Operator rancher-backup:104.0.1+up5.0.1

Describe the bug If you define *.* in the resources of a EncryptionConfiguration for an encrypted backup, nothing gets encrypted. Is this intended behaviour?

To Reproduce

1. Create a EncryptionConfiguration:

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
      - "*.*"
    providers:
      - secretbox:
          keys:
            - name: key1
              secret: <REDACTED>

2. Create secret with this file as content.

   kubectl create secret generic encryptionconfig \
   --from-file=./encryption-provider-config.yaml \
   -n cattle-resource-system

3. Create a encrypted Rancher backup with the backup operator.
4. Download the backup, (rename to tar.gz and) extract, inspect the resource json's
5. None of the resource json are encrypted.

Expected behavior All the resources json are encrypted (like when you use secrets as resource.

[bvankampen]

Using * instead of *.* doesn't work too.

[mallardduck]

Identified the source of the bug to around here: https://github.com/rancher/backup-restore-operator/blob/5cc7bccaeaa6fcb44a8bc8ede18a53dfbe1ad9f6/pkg/resourcesets/collector.go#L390

Gist being that it only currently allows direct matches and cannot work with wildcards. Looking into potential solutions.

[mallardduck]

After further investigation, I've realized that the upstream ability to use this feature (wildcard resource selection for EncryptionConfiguration) is something that requires k8s 1.27+.

The k8s docs indicate this in their examples:

      - '*.*' # wildcard match requires Kubernetes 1.27 or later

So this is more of an RFE rather than an outright bug. The main reason being that BRO releases are tied to Rancher releases and must support the same range of k8s versions. So ideally a feature like this is included when Rancher minimum k8s matches the version that upstream adds the feature in.

Thankfully in this case that lines up with 2.9.x Rancher - so there's no "bake in" time for that window to expire. I have been able to make a lot of progress to be confident on how we can fix backups. However the fix needs to be symmetrical or it will leave restorations broken.

---

Dev notes: To fix we will want to more directly replicate the logic k8s uses when dealing with EncryptionConfiguration. The glaring issue is that we use keys to find the transformers to use and this wildcard feature requires a less direct approach.

Because the wildcard feature allows for: true wildcards (*.*), core wildcards (*.), partial wildcards (*.apps) and for those to use the no operation (identity) provider. This mix of new combinations means that: a) we need to take that in mind when resolving transformers and b) even unencrypted files will likely have transformers now.

Attached is a backup-encrypt.patch file that fixes uploads, but as mentioned a symmetrical fix to restores must be applied as well. Likely this is a M or L task.

[mallardduck]

The PR work on this is complete but given that it targets Rancher 2.11 it is blocked until after 2.10 release.

[jbiers]

We managed to get QA cycles to get this tested for the 2.10 release so I'll be merging the PR and moving the issue from blocked ;)

[jbiers]

QA notes: Unit and integration testing for this feature were added in a separate PR. Maybe it is part of standard procedure to use the latest RC version, but I'm commenting this here just to make sure. Latest version including the updated tests is 105.0.0+up6.0.0-rc.7

[nickwsuse]

Verified on v2.10-alpha ID: 212d8b6 & Rancher Backups v105.0.0+up6.0.0-rc.7 ✅

Verified on fresh install of v2.10-head and upgrades from v2.9.3 the encryption config correctly encrypts the right resources. If specific resources are given in the config, or if the wildcard operator is used, encryption is working for backups and for restores.

resources:
  - resources:
      - secrets
      - configmaps

resources:
  - resources:
      - "*.*"