search

gardener / etcd-backup-restore

Collection of components to backup and restore the etcd of a Kubernetes cluster.
Apache License 2.0
289 stars 100 forks source link

[Enhancement] Restoration failed due to full-snapshot taken by backup-restore is found to be corrupted or not having a hash #778

Open ishan16696 opened 2 months ago

ishan16696 commented 2 months ago

Enhancement (What you would like to be added): It has been observed that while doing the restoration from full snapshot, backup-restore failed to restore the etcd from full-snapshot. This failure appears to be due to some full snapshots being corrupted or missing a hash.

Logs of backup-restore during restoration.

FATA[0002] Failed to restore snapshot: failed to restore from the base snapshot: snapshot missing hash but --skip-hash-check=false

Motivation (Why is this needed?):

  1. One of such case is reported here: https://github.com/gardener/etcd-backup-restore/issues/749 by community user.
  2. We have also observed such cases in our internal clusters while doing the recovery from permanent quorum loss for etcd cluster.

Approach/Hint to the implement solution (optional):

Screenshot 2024-09-23 at 10 10 45 AM
ishan16696 commented 2 months ago

/assign