search

Azure / aksArc

# Welcome to the Azure Kubernetes Service enabled by Azure Arc (AKS Arc) repo This is where the AKS Arc team will track features and issues with AKS Arc. We will monitor this repo in order to engage with our community and discuss questions, customer scenarios, or feature requests. Checkout our projects tab to see the roadmap for AKS Arc!
MIT License
111 stars 45 forks source link

[BUG] Velero backup job gives errors #179

Open jessicaguan opened 2 years ago

jessicaguan commented 2 years ago

Describe the bug Customer was able to take the backup successfully, but a few days later they ran another backup job in Velero without any changes. The backup gave error messages.

To Reproduce Steps to reproduce the behavior: Followed instructions in the doc: https://docs.microsoft.com/en-us/azure-stack/aks-hci/backup-workload-cluster#:~:text=This%20topic%20describes%20how%20to%20back%20up%20and,variety%20of%20storage%20providers%20to%20store%20its%20backups.

Expected behavior Velero to complete backup job successfully.

Screenshots time="2022-03-19T19:37:53Z" level=info msg="1 errors encountered backup up item" backup=velero/frstbck logSource="pkg/backup/backup.go:412" name=csi-akshcicsi-controller-5c7d5b49-f42q9 time="2022-03-19T19:37:53Z" level=error msg="Error backing up item" backup=velero/frstbck error="timed out waiting for all PodVolumeBackups to complete" error.file="/go/src/github.com/vmware-tanzu/velero/pkg/restic/backupper.go:177" error.function="github.com/vmware-tanzu/velero/pkg/restic.(*backupper).BackupPodVolumes" logSource="pkg/backup/backup.go:416" name=csi-akshcicsi-controller-5c7d5b49-f42q9

time="2022-03-19T19:37:53Z" level=info msg="1 errors encountered backup up item" backup=velero/frstbck logSource="pkg/backup/backup.go:412" name=csi-akshcicsi-node-4kzmc time="2022-03-19T19:37:53Z" level=error msg="Error backing up item" backup=velero/frstbck error="timed out waiting for all PodVolumeBackups to complete" error.file="/go/src/github.com/vmware-tanzu/velero/pkg/restic/backupper.go:177" error.function="github.com/vmware-tanzu/velero/pkg/restic.(*backupper).BackupPodVolumes" logSource="pkg/backup/backup.go:416" name=csi-akshcicsi-node-4kzmc

time="2022-03-19T19:37:53Z" level=info msg="1 errors encountered backup up item" backup=velero/frstbck logSource="pkg/backup/backup.go:412" name=moc-cloud-controller-manager-5fbb78df68-4qg48 time="2022-03-19T19:37:53Z" level=error msg="Error backing up item" backup=velero/frstbck error="timed out waiting for all PodVolumeBackups to complete" error.file="/go/src/github.com/vmware-tanzu/velero/pkg/restic/backupper.go:177" error.function="github.com/vmware-tanzu/velero/pkg/restic.(*backupper).BackupPodVolumes" logSource="pkg/backup/backup.go:416" name=moc-cloud-controller-manager-5fbb78df68-4qg48

time="2022-03-19T19:37:54Z" level=info msg="1 errors encountered backup up item" backup=velero/frstbck logSource="pkg/backup/backup.go:412" name=restic-vszgv time="2022-03-19T19:37:54Z" level=error msg="Error backing up item" backup=velero/frstbck error="timed out waiting for restic repository to become ready" error.file="/go/src/github.com/vmware-tanzu/velero/pkg/restic/repository_ensurer.go:183" error.function="github.com/vmware-tanzu/velero/pkg/restic.(*repositoryEnsurer).EnsureRepo" logSource="pkg/backup/backup.go:416" name=restic-vszgv

time="2022-03-19T19:37:54Z" level=info msg="1 errors encountered backup up item" backup=velero/frstbck logSource="pkg/backup/backup.go:412" name=velero-b557bf84f-hrvzd time="2022-03-19T19:37:54Z" level=error msg="Error backing up item" backup=velero/frstbck error="restic repository is not ready: " error.file="/go/src/github.com/vmware-tanzu/velero/pkg/restic/repository_ensurer.go:144" error.function="github.com/vmware-tanzu/velero/pkg/restic.(*repositoryEnsurer).EnsureRepo" logSource="pkg/backup/backup.go:416" name=velero-b557bf84f-hrvzd

Environment (please complete the following information):