Establish contract between kube-proxy on Windows and CNIs for handling source VIPs in overlay networking mode

tzifudzi commented 9 months ago

What would you like to be added?

Introduce a well-known, documented name for the HNS endpoint used as the source VIP, proposed to be named sourcevip. When kube-proxy starts up in VXLAN overlay networking mode, it will check for the existence of an HNS endpoint named sourcevip. If found, the IP of this HNS endpoint will be used as the source VIP.
The precedence will be as follows:
1. Is sourceVip specified in the KubeProxyWinkernelConfiguration? If yes, use it. This step already exists.
2. Does an HNS endpoint exist with the well-known name? If yes, use its IP as the sourceVip. This step is the proposed change.
3. Fallback to creating a new source VIP. If this fails, log an error. This step already exists.
Scope clarification: This change applies to Windows nodes operating in overlay networking mode.

Why is this needed?

Currently, in overlay networking mode on Windows nodes, kube-proxy can be configured with a source VIP. The challenge arises because the IP address for the source VIP is not always predetermined and varies depending on the CNI implementation. A well-known HNS endpoint name would streamline the process by allowing the CNI to set the source VIP on a well known name, which kube-proxy can then reliably identify.
Various CNIs have developed workarounds for this issue. This proposal aims to eliminate the need for such CNI-specific solutions, thereby enhancing ease of use and maintainability. For instance, Flannel utilizes a script that must run before starting kube-proxy to determine the source VIP. This change would therefore render such scripts unnecessary.
This change is also of importance in light of ongoing efforts in sig-windows to consolidate the kube-proxy image into an official host process Kubernetes container image, moving away from the currently more popular way of running kube-proxy as a binary agent on Windows nodes. A unified contract between kube-proxy and CNIs would facilitate the use of a this kube-proxy image across different CNIs.

### Tasks
- [x] Create draft PR and get early feedback from CNIs that support overlay networking mode
- [ ] Update kubernetes documentation with information on the well known HNS endpoint name
- [ ] Add unit tests
- [ ] Perform code changes and convert draft to final PR

References

sig-windows Google Doc discussing this change

tzifudzi commented 9 months ago

/kind feature /sig windows /milestone v1.30

k8s-ci-robot commented 9 months ago

@tzifudzi: You must be a member of the kubernetes/milestone-maintainers GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your Milestone Maintainers Team and have them propose you as an additional delegate for this responsibility.

In response to [this](https://github.com/kubernetes/kubernetes/issues/123014#issuecomment-1913939777): >kind/feature >sig/windows >/milestone v1.30 Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

tzifudzi commented 9 months ago

/kind feature /sig windows

cc @jsturtevant

tzifudzi commented 9 months ago

/assign tzifudzi

tzifudzi commented 9 months ago

/area kube-proxy

pranav-pandey0804 commented 8 months ago

hi @tzifudzi , I appreciate you bringing up the need to establish a clear protocol between kube-proxy on Windows and CNIs for handling source VIPs. This is a crucial step forward in enhancing interoperability and simplifying configurations. Thank you for your efforts in addressing this issue.

pranav-pandey0804 commented 8 months ago

@tzifudzi I kindly suggest integrating a validation feature within kube-proxy to confirm CNI compliance with the proposed naming standards for source VIPs in overlay networks. This proactive measure could enhance integration and prevent potential misconfigurations, ensuring a smoother collaboration between kube-proxy and CNIs.

pranav-pandey0804 commented 8 months ago

I kindly suggest integrating a validation feature within kube-proxy to confirm CNI compliance with the proposed naming standards for source VIPs in overlay networks

however, implementing it could increase complexity, diverse CNI architectures, ensuring backward compatibility, and crafting precise error handling strategies. These aspects would need careful navigation to integrate effectively.

pranav-pandey0804 commented 8 months ago

@tzifudzi I'm genuinely interested in contributing to the conversation and development of this feature. Should there be a need for deeper discussion or hands-on assistance in bringing this concept to fruition, I'm more than willing to lend my expertise and collaborate closely. Thanks for the contribution!

jsturtevant commented 8 months ago

/triage accepted /priority important-longterm

Adding the original document at was shared with sig-windows for historical reference: https://docs.google.com/document/d/1A6Gkyx5EvL86Z4L2P4sxnk9HQjdRgnLXvBg6tIMDB3s/edit#heading=h.rqiiboge3f2e

jsturtevant commented 8 months ago

/cc @daschott

tzifudzi commented 8 months ago

@pranav-pandey0804 Thanks for the comments and considerations. In the notes in the description I briefly touched on the error handling workflow and considerations for backward compatibility. Will add more information in the PR when the change is ready. I will be working on this change alongside jsturtevant@ and assigned the issue to myself, but we will need all the feedback we can get. I will add you as a reviewer on the PR.

pranav-pandey0804 commented 8 months ago

Thanks, @tzifudzi, for considering my input and the detailed update. I'm keen to see how we tackle error handling and backward compatibility in the PR. Looking forward to contributing to the review process.

kubernetes / kubernetes