[Release-1.26] - `ListObjectsV2 search parameter metadata not implemented` when using Cloudflare R2

[brandond]

Backport fix for ListObjectsV2 search parameter metadata not implemented when using Cloudflare R2

• 8924

[VestigeJ]

Reproduced here - https://github.com/k3s-io/k3s/issues/8924#issuecomment-1823315855

Environment Details

Validated using VERSION=v1.26.11-rc1+k3s1

Infrastructure

• [X] Cloud

Node(s) CPU architecture, OS, and version:

Linux 5.14.21-150500.53-default x86_64 GNU/Linux PRETTY_NAME="SUSE Linux Enterprise Server 15 SP5"

Cluster Configuration:

NAME           STATUS   ROLES                       AGE     VERSION
ip-1-26-1-26   Ready    control-plane,etcd,master   7m30s   v1.26.11-rc1+k3s1

Config.yaml:

write-kubeconfig-mode: 644
debug: true
token: YOUR_TOKEN_HERE
profile: cis
selinux: true
node-external-ip: 1.............................
protect-kernel-defaults: true
cluster-init: true

etcd-s3: true
etcd-s3-bucket: "k3s-etcd-testing"
etcd-s3-endpoint: "bwop.r2.cloudflarestorage.com"
etcd-s3-access-key: "bwaaab"
etcd-s3-secret-key: "bwodd"

YOUR_REPRODUCED_RESULTS_HERE

``` $ curl https://get.k3s.io --output install-"k3s".sh $ sudo chmod +x install-"k3s".sh $ sudo groupadd --system etcd && sudo useradd -s /sbin/nologin --system -g etcd etcd $ sudo modprobe ip_vs_rr $ sudo modprobe ip_vs_wrr $ sudo modprobe ip_vs_sh $ sudo printf "on_oovm.panic_on_oom=0 \nvm.overcommit_memory=1 \nkernel.panic=10 \nkernel.panic_ps=1 \nkernel.panic_on_oops=1 \n" > ~/90-kubelet.conf $ sudo cp 90-kubelet.conf /etc/sysctl.d/ $ sudo systemctl restart systemd-sysctl $ sudo INSTALL_K3S_VERSION=v1.26.11-rc1+k3s1 INSTALL_K3S_EXEC=server ./install-k3s.sh $ w2 kg no,po -A //watch -n2 kubectl get nodes,pods -A $ sudo /usr/local/bin/k3s etcd-snapshot save //observe r2 bucket for snapshot files ``` **Results:** 

$ sudo /usr/local/bin/k3s etcd-snapshot save

sudo /usr/local/bin/k3s etcd-snapshot save --name twentysix
WARN[0000] Unknown flag --write-kubeconfig-mode found in config.yaml, skipping
WARN[0000] Unknown flag --token found in config.yaml, skipping
WARN[0000] Unknown flag --profile found in config.yaml, skipping
WARN[0000] Unknown flag --selinux found in config.yaml, skipping
WARN[0000] Unknown flag --node-external-ip found in config.yaml, skipping
WARN[0000] Unknown flag --protect-kernel-defaults found in config.yaml, skipping
WARN[0000] Unknown flag --cluster-init found in config.yaml, skipping
DEBU[0000] Attempting to retrieve extra metadata from k3s-etcd-snapshot-extra-metadata ConfigMap
DEBU[0000] Error encountered attempting to retrieve extra metadata from k3s-etcd-snapshot-extra-metadata ConfigMap, error: configmaps "k3s-etcd-snapshot-extra-metadata" not found
INFO[0000] Saving etcd snapshot to /var/lib/rancher/k3s/server/db/snapshots/twentysix-ip-1-26-1-26-10364
{"level":"info","ts":"2023-11-27T18:39:23.622741Z","caller":"snapshot/v3_snapshot.go:65","msg":"created temporary db file","path":"/var/lib/rancher/k3s/server/db/snapshots/twentysix-ip-1-26-1-26-10364.part"}
{"level":"info","ts":"2023-11-27T18:39:23.625309Z","logger":"client","caller":"v3@v3.5.9-k3s1/maintenance.go:212","msg":"opened snapshot stream; downloading"}
{"level":"info","ts":"2023-11-27T18:39:23.625451Z","caller":"snapshot/v3_snapshot.go:73","msg":"fetching snapshot","endpoint":"https://127.0.0.1:2379"}
{"level":"info","ts":"2023-11-27T18:39:23.656547Z","logger":"client","caller":"v3@v3.5.9-k3s1/maintenance.go:220","msg":"completed snapshot read; closing"}
{"level":"info","ts":"2023-11-27T18:39:23.666745Z","caller":"snapshot/v3_snapshot.go:88","msg":"fetched snapshot","endpoint":"https://127.0.0.1:2379","size":"3.3 MB","took":"now"}
{"level":"info","ts":"2023-11-27T18:39:23.666813Z","caller":"snapshot/v3_snapshot.go:97","msg":"saved","path":"/var/lib/rancher/k3s/server/db/snapshots/twentysix-ip-1-26-1-26-10364"}
INFO[0000] Checking if S3 bucket k3s-etcd-testing exists
INFO[0000] S3 bucket k3s-etcd-testing exists
INFO[0000] Saving etcd snapshot twentysix-ip-1-26-1-26-10364 to S3
INFO[0000] Uploading snapshot to s3://k3s-etcd-testing//var/lib/rancher/k3s/server/db/snapshots/twentysix-ip-1-26-1-26-10364
INFO[0001] Uploaded snapshot metadata s3://k3s-etcd-testing/.metadata/twentysix-ip-1-26-1-26-10364
INFO[0001] S3 upload complete for twentysix-ip-1-26-1-26-10364
INFO[0001] Reconciling ETCDSnapshotFile resources
DEBU[0001] Found snapshotFile for twentysix-ip-1-26-1-26-10364 with key local-twentysix-ip-1-26-1-26-10364
DEBU[0001] Found snapshotFile for twentysix-ip-1-26-1-26-10364 with key s3-twentysix-ip-1-26-1-26-10364
DEBU[0001] Found ETCDSnapshotFile for twentysix-ip-1-26-1-26-10364 with key local-twentysix-ip-1-26-1-26-10364
DEBU[0001] Found ETCDSnapshotFile for twentysix-ip-1-26-1-26-10364 with key s3-twentysix-ip-1-26-1-26-10364
INFO[0001] Reconciliation of ETCDSnapshotFile resources complete