Windows node scheduling

[grampelberg]

What problem are you trying to solve?

As of today, the proxy does not run on windows nodes. Unfortunately, injection will still happen and the pod will just not run.

How should the problem be solved?

• Update inject to be aware of windows pods and skip injection on them.
• Update check to warn if there are windows nodes in the cluster.

Open questions

• Are there any ways we can surface that a pod is failing because of the proxy in a clear way?
• Should we recommend taints in the documentation?

[vaniisgh]

Hey!

If this issue is still relevant, I'd like to have a go at it :) thanks

[grampelberg]

AFAIK it is still relevant!

[grampelberg]

We need to be adding the scheduling label as well so that proxies aren't scheduled to windows nodes in mixed clusters.

[grampelberg]

@alpeb did we ever land the change for the proxy injector to fail injection requests when it can't?

[alpeb]

@grampelberg Nope, that still hasn't landed #4235

[vaniisgh]

Should this issue be tackled only after that one has been merged since upon a cursory glance it seems to me that there are parts of code from that PR that could be extended to cover this one like the GetInjectFailReason could be made aware of windows pods and skip them while logging a WARNING?

I figure that check can still be updated to give warnings without causing any conflicts with that branch :)

adding scheduling labels sounds interesting, I don't know much about that I will certainly look into that as well if that's ok?

thanks :)

[grampelberg]

@vaniisgh they're kinda separate as this is just adding more reasons we can't inject a pod. So both this and the scheduling labels sounds like a great thing to work on!

[vaniisgh]

Oh! my bad... they looked similar to the untrained eye I guess :) Thanks for the clarification. Sounds good.

[mayankshah1607]

@vaniisgh Let me know if you want to give #4235 a try. I'd be happy to help. :)

[vaniisgh]

@mayankshah1607 really! thank you so much! I just spent so long tracing the functions to find where I actually need to add code for this one, I think I have it figured. for #4235 you did quite a bit so that would be super helpful! I also have a few very basic questions... would it be ok if I ping you on slack :)?

[mayankshah1607]

@vaniisgh Sure, feel free to do so!

[Type1J]

Is there an effort to try to make the proxy run on Windows? I have a few projects that would benefit.

[cpretzer]

@Type1J I haven't heard of anyone working on this since @vaniisgh took it up, and to clarify, the purpose of this issue is to prevent the proxy from being injected into workloads running on Windows nodes.

To answer your question, I believe some of the Deis/Microsoft team were working on this a while back, but I haven't heard anything recently.

If I remember correctly, the main issue with running the proxy on Windows nodes is the traffic routing. The proxy-init container uses logic to modify iptables in order to proxy the traffic.

Since Windows doesn't use iptables the proxy-init container can't route the traffic. To say that another way, the work involved to run the proxy on Windows includes detecting that the underlying OS is Windows and programmatically creating the firewall rules.

If you're familiar with Windows programming, we'd love your help with this. 😄

[Type1J]

iptables ... I think that's Window's biggest problem for service meshes. I'm hoping that somebody else will solve this problem on Windows before I'm forced into it.

[cpretzer]

You're our only hope @Type1J 😄

[Type1J]

I'm still too busy. I've sent quite a few .Net Framework projects off to be remade in .Net 5, 6, and 7 to be runnable on a Linux node, so we can include it in a service mesh.