search

linkerd / linkerd2

Ultralight, security-first service mesh for Kubernetes. Main repo for Linkerd 2.x.
https://linkerd.io
Apache License 2.0
10.62k stars 1.27k forks source link

controller to roll pods that are not injected but should be #4015

Open derrickburns opened 4 years ago

derrickburns commented 4 years ago

Feature Request

One the the challenges of using Linkerd auto-injection is dealing with silent failure. How about taking the K8s approach of controllers that continuously work to achieve a target state? In this case, I suggest a controller the continuously scans all pods and namespaces with the linkerd injection annotation. When the controller finds a pod that has the annotation but does not have the proper linkerd sidecar, it restarts the pod.

The alternative to this is some set of alerts that each client must hand craft and then respond to.

One could make this controller an optional component.

grampelberg commented 4 years ago

@derrickburns are you running in --ha? It shouldn't be possible to have pods without the sidecar in that case.

derrickburns commented 4 years ago

@grampelberg No, I am not.

Consider these cases:

Case 1:

I tried to update the Linkerd control plane. However, I also updated the ca. Which means that all the sidecars need to be re-injected. So, even if sidecars are present, the pod identity may not be correct.

Case 2: Cluster auto-scaler moves pod. New pod is not auto-injected.

I have definitely hit case 1 and I vaguely recall hitting case 2.

grampelberg commented 4 years ago

We've had a lot of discussions around case 1. The concern has been that rolling workloads can be a pretty big deal for most folks. Getting an operator as an opt-in solution seems like a good idea though.

--ha should solve case 2 for you.

derrickburns commented 4 years ago

One could also add an annotation to namespaces/pods to opt-in or opt-out of auto-rolling.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

ko185061 commented 2 years ago

Are there any current issues related to this or any progress that has been made? This functionality would be very helpful, thanks in advance!

olix0r commented 2 years ago

@ko185061 Nothing in Linkerd does this. But, this sort of functionality wouldn't necessarily have to be Linkerd-specific. There may be other controllers out there that implement this sort of functionality (gradually restarting pods that don't satisfy some label/annotation condition). I'm not sure of any off the top my head, though.