search

vmware-tanzu / velero

Backup and migrate Kubernetes applications and their persistent volumes
https://velero.io
Apache License 2.0
8.69k stars 1.4k forks source link

Inherited maintenance job annotations prevents job from completing #8049

Open wolffberg opened 3 months ago

wolffberg commented 3 months ago

What steps did you take and what happened: As of version v1.14 Velero maintenance jobs inherit all annotations from the Velero deployment. When using sidecar containers for the deployment (e.g. HashiCorp Vault Sidecar Injector) the sidecars are also added to the jobs which then never completes as the sidecars keep the job alive.

In above case you would normally add the vault-hashicorp-com-agent-pre-populate-only annotation to the jobs.

What did you expect to happen: Annotations should be configurable for the maintenance jobs.

Environment:

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

blackpiglet commented 3 months ago

https://github.com/vmware-tanzu/velero/blob/main/design/repo_maintenance_job_config.md There is a design trying to make the repository maintenance job more configurable. I think we can add one more section for the annotations.

wolffberg commented 3 months ago

That would indeed be a viable solution. Can we track progress and ETA of having the design released?

blackpiglet commented 3 months ago

The design is merged and the implementation should be in the v1.15.0.

blackpiglet commented 3 months ago

This requirement will not be addressed in v1.15. Need to give more consideration to how to generate the annotations.

For example, some cloud providers need the annotation containing authorization information for the service account scenario.

wolffberg commented 2 months ago

Can you give some examples on how it would be an issue?

If you made all annotations configurable for jobs wouldn't that also fix any required authorization annotations?

blackpiglet commented 2 months ago

I got your point, but if the annotations are all configurable, users need to configure the annotation to include the IRSA or other kinds of authentication information to make it work.

Users could get confused about how to fix that.

wolffberg commented 2 months ago

For IRSA the annotations are only added to the ServiceAccount so as long as the job inherits the ServiceAccount that should still function as expected.

If there are any concerns about jobs needing attributes from the Velero pod, you could also make the jobs inherit from Velero and then make it overwrite any existing attributes from the ConfigMap.

That way nothing would be changed by default but would allow us (in this case) to set vault.hashicorp.com/agent-pre-populate-only=true.

You could then make a switch to fully disable attribute inheritence or just make any null values completely remove it from the job.