velero pvc and pv snapshots fails during backup

[gcasali01]

I'm trying to perform tanzu Kubernetes cluster backups following this guide

I'm using velero client and server same versions : v1.13.1

Client:
        Version: v1.13.1
        Git commit: ea5a89f83b89b2cb7a27f54148683c1ee8d57a37
Server:
        Version: v1.13.1

I installed velero on my Kubernetes cluster in this way and I'm using minio as object storage :

velero install \
    --provider aws \
    --plugins velero/velero-plugin-for-aws:v1.9.0,velero/velero-plugin-for-csi:v0.7.0 \
    --bucket tanzu \
    --secret-file s3-credentials \
    --use-volume-snapshots=true \
    --features=EnableCSI --use-node-agent \
    --snapshot-location-config region=minio \
    --backup-location-config region=minio,s3ForcePathStyle="true",s3Url=https://minio-url.net --cacert ./ca.crt

The tanzu kubernetes release is this one : v1.26.5+vmware.2-fips.1 When I try to perform the cluster backup pvc backups and pvc snapshots are not performed and I got this error :

<error getting backup volume info: DownloadRequest.velero.io "cluster01-bkp12-b3a7f665-3527-47a4-b559-e7397e31dfe4" is invalid: spec.target.kind: Unsupported value: "BackupVolumeInfos": supported values: "BackupLog", "BackupContents", "BackupVolumeSnapshots", "BackupItemOperations", "BackupResourceList", "BackupResults", "RestoreLog", "RestoreResults", "RestoreResourceList", "RestoreItemOperations", "CSIBackupVolumeSnapshots", "CSIBackupVolumeSnapshotContents">

If using the vsphere CSI I'm able to create pv snapshots without velero.

[sseago]

It looks like you have old CRDs installed. "BackupVolumeinfos" is a newly-added value for DownloadRequest spec.target.kind.

[gcasali01]

hi all , thanks for replying , I redeployed velero by removing all velero resources and CRDs , now I have another strange error due to TLS timeout :

Backup Volumes:
  <error getting backup volume info: Get "https://minio-url.net/tanzu/backups/cluster01-bkp13/cluster01-bkp13-volumeinfo.json.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=tanzuak%2F20240322%2Fminio%2Fs3%2Faws4_request&X-Amz-Date=20240322T144604Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=341e932c556757e062d7eebbe575313f9c5bfc6033753f0749ec66c335f39375": net/http: TLS handshake timeout>

minio was populated by velero , so network reachbility is in place. This is the full backup describe :

 velero backup describe cluster01-bkp13
Name:         cluster01-bkp13
Namespace:    velero
Labels:       velero.io/storage-location=default
Annotations:  velero.io/resource-timeout=10m0s
              velero.io/source-cluster-k8s-gitversion=v1.26.5+vmware.2-fips.1
              velero.io/source-cluster-k8s-major-version=1
              velero.io/source-cluster-k8s-minor-version=26

Phase:  Completed

Namespaces:
  Included:  *
  Excluded:  <none>

Resources:
  Included:        *
  Excluded:        <none>
  Cluster-scoped:  auto

Label selector:  <none>

Or label selector:  <none>

Storage Location:  default

Velero-Native Snapshot PVs:  auto
Snapshot Move Data:          false
Data Mover:                  velero

TTL:  720h0m0s

CSISnapshotTimeout:    10m0s
ItemOperationTimeout:  4h0m0s

Hooks:  <none>

Backup Format Version:  1.1.0

Started:    2024-03-22 15:28:56 +0100 CET
Completed:  2024-03-22 15:30:29 +0100 CET

Expiration:  2024-04-21 16:28:56 +0200 CEST

Total items to be backed up:  1174
Items backed up:              1174

Backup Item Operations:  14 of 14 completed successfully, 0 failed (specify --details for more information)
Backup Volumes:
  <error getting backup volume info: Get "https://minio-url.net/tanzu/backups/cluster01-bkp13/cluster01-bkp13-volumeinfo.json.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=tanzuak%2F20240322%2Fminio%2Fs3%2Faws4_request&X-Amz-Date=20240322T144604Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=341e932c556757e062d7eebbe575313f9c5bfc6033753f0749ec66c335f39375": net/http: TLS handshake timeout>

HooksAttempted:  0
HooksFailed:     0

[blackpiglet]

@gcasali01 Please try to add the parameter --cacert or --insecure-skip-tls-verify in the velero backup describe CLI. After v1.13.0, the velero backup describe CLI reads the volume information metadata file from the backup repository to display the volumes included in the backup.