migtools / oadp-bugs

0 stars 0 forks source link

DPA config for maint jobs #3

Open weshayutin opened 3 weeks ago

weshayutin commented 3 weeks ago

The Velero maintain jobs to compact the kopia repositories are running with no memory or cpu limits defined. We tracked down that these are set via the Velero command line options. This is OADP 1.4/Velero 1.14. --maintenance-job-cpu-request --maintenance-job-mem-request --maintenance-job-cpu-limit --maintenance-job-mem-limit Which are typically set by the dataprotectionapplication object in spec.configuration.velero.args. These cannot be set as the Velero deployment values are reset due to being controlled by OADP. The only option for maintain jobs found under args "default-repo-maintain-frequency". Also, I can tell from our testing this needs to be rather large, possibly similar to the datamover resource usage. Our larger CET tests replicating CloudPak4Data worst PVC is only succeeding because it using unlimited resource requests/limits right now. Openshift picked up a memory usage of ~800MB, but it is has been several days since the last backup of this one and we know these values are worse recently after a backup than later. Figuring out their memory usage isn't easy because these jobs typically are lasting <30 seconds meaning not often picked up by the Openshift Dashboard at all.