vmware-tanzu / velero

Backup and migrate Kubernetes applications and their persistent volumes
https://velero.io
Apache License 2.0
8.78k stars 1.41k forks source link

Allow Ephemeral Storage Customization for backupPod in Velero #8317

Open shubham-pampattiwar opened 1 month ago

shubham-pampattiwar commented 1 month ago

Describe the problem/challenge you have

Currently, the ephemeral storage allocated to Velero backupPod is not configurable, which may cause problems when handling large backups or complex workloads. It can lead to Pod evictions, Operational Failures or Performance degradation, Disk pressure events etc.

Describe the solution you'd like

Enable customization of ephemeral storage requests and limits in the backupPod configuration, similar to the way CPU and memory limits are adjustable.

Anything else you would like to add:

Environment:

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

Lyndon-Li commented 1 month ago

backupPod/restorePod doesn't use ephemeral disk space except the data/metadata cache for Kopia repo. For repo cache, the plan is to allow a customized location to store the cache data, see #7725.

And I don't think the backupPod/restorePod is related to the workload's complexity since the backupPod/restorePod is only used to move data of volumes.

Therefore, it is not necessary to have users configure the pod's ephemeral space in a general way.

Moreover, users should control data mover from end to end, while backupPod/restorePod is not exposed to/known by users directly. Instead, it is highly programmed based on the e2e requirements, and so is delicate because it needs to support various scenarios. So unless it is necessary and we know why, we should not add a configuration by considering backupPod/restorePod as general pods.

s4ndalHat commented 1 month ago

Hello, Facing the same issue with kopia uploader. As a workaround for now, we use the deprecated restic uploader. It works, but sometimes encounters restic lock behaviour.

I already got answer from one of Velero's maintainers that this would be on 1.15 milestone fix or maybe later, Looking forward for this feature,

Thanks in advance to the Velero team!

Best regards,