Multi-goroutine backup/restore

[Hysteria23]

problem/challenge

The current BackupWithResolvers function in Velero processes backup items sequentially. This serial processing can be a performance bottleneck, especially for large Kubernetes clusters with many resources. It results in longer backup times, which can impact the overall efficiency and reliability of backup operations in large-scale environments.

Solution proposal

Modify the BackupWithResolvers function to process backup items concurrently using goroutines. This will involve:

1. Creating a worker pool to manage the concurrent processing of backup items.
2. Using channels to communicate between the main backup loop and the worker goroutines.

Environment:

• Velero version (use velero version): v1.14.0
• Kubernetes version (use kubectl version): v1.29
• Kubernetes installer & version: kubeadm 1.29
• Cloud provider: AWS,Azure,GCP
• OS: Ubuntu 20.04

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

• 👍 for "The project would be better with this feature added"
• 👎 for "This feature will not enhance the project in a meaningful way"

[Lyndon-Li]

7628 this design will cover this requirement

[kaovilai]

Close as dupe of https://github.com/vmware-tanzu/velero/issues/7900 ?