openyurtio / openyurt

OpenYurt - Extending your native Kubernetes to edge(project under CNCF)
https://openyurt.io
Apache License 2.0
1.7k stars 401 forks source link

[feature request] Set K8s service host & port to Yurthub Proxy Host & Port in Yurthub Component #2158

Open techworldhello opened 5 days ago

techworldhello commented 5 days ago

What would you like to be added:

A filter in Yurthub that sets the KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT environment variables to the Yurthub proxy host and IP. This enhancement would ensure node autonomy for edge nodes when they are operating offline.

Why is this needed:

Some managed services may use an admission webhook to modify the KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT environment variables, setting them to the public FQDN and port of the kube-apiserver. For these managed services to operate with Yurthub, the pods scheduled on edge nodes must be directed to Yurthub so that requests to the kube-apiserver can be proxied through it.

We are requesting the implementation of a filter that would ensure all edge pods access the kube-apiserver via the Yurthub proxy. This filter would add logic without disrupting other users. If a user prefers to disable this filter in the future, they could do so by adding it to the --disabled-resource-filters startup parameter.

others /kind feature

zyjhtangtang commented 3 days ago

masterServiceFilter has already implemented this capability. By default, Yurthub only caches request data for system components. If the business requires utilizing Yurthub's cache to operate during network outages, the corresponding User-Agent (UA) needs to be configured in the ConfigMap yurt-hub-cfg. reference: https://github.com/openyurtio/openyurt/blob/master/docs/proposals/20220627-yurthub-cache-refactoring.md#41-enable-yurthub-to-distinguish-resources-with-same-name-but-different-versions-and-groups

rambohe-ch commented 3 days ago

masterServiceFilter has already implemented this capability. By default, Yurthub only caches request data for system components. If the business requires utilizing Yurthub's cache to operate during network outages, the corresponding User-Agent (UA) needs to be configured in the ConfigMap yurt-hub-cfg. reference: https://github.com/openyurtio/openyurt/blob/master/docs/proposals/20220627-yurthub-cache-refactoring.md#41-enable-yurthub-to-distinguish-resources-with-same-name-but-different-versions-and-groups

@zyjhtangtang I think that masterservice Filter can not solve the problem that mentioned in the issue. because masterservice filter only modify default/kubernetes service, this means that pods without KUBERNETES_SERVICE_HOST env can work with Yurthub component seamlessly. but pods with KUBERNETES_SERVICE_HOST can not work together with Yurthub component. maybe it is a good idea to overwrite KUBERNETES_SERVICE_HOST env of pod by Yurthub proxy address on edge nodes.

rambohe-ch commented 3 days ago

What would you like to be added:

A filter in Yurthub that sets the KUBERNETES_SERVICE_HOST environment variable to the Yurthub proxy IP. This enhancement would ensure node autonomy for edge nodes when they are operating offline.

Why is this needed:

Some managed services may use an admission webhook to change the KUBERNETES_SERVICE_HOST environment variable to the public FQDN of the kube-apiserver. For these managed services to operate with Yurthub, the pods scheduled on edge nodes must be directed to Yurthub so that requests to the kube-apiserver can be proxied through it.

We are requesting the implementation of a filter that would ensure all edge pods access the kube-apiserver via the Yurthub proxy. This filter would add logic without disrupting other users. If a user prefers to disable this filter in the future, they could do so by adding it to the --disabled-resource-filters startup parameter.

others /kind feature

@techworldhello Thank you for raising the issue. Only pods without KUBERNETES_SERVICE_HOST env can access kube-apiserver through yurthub currently. so it seems that a new filter is necessary for dealing with pods with KUBERNETES_SERVICE_HOST env on edge nodes.

zyjhtangtang commented 2 days ago

@techworldhello @rambohe-ch Yes, if the pod has already been set with KUBERNETES_SERVICE_HOST, the masterservice filter will not take effect. In this case, It is necessary to add a new filter to directly modify it.

techworldhello commented 1 day ago

@zyjhtangtang @rambohe-ch I've updated this issue to link to a new PR https://github.com/openyurtio/openyurt/pull/2165, as it made sense to update KUBERNETES_SERVICE_PORT as well as KUBERNETES_SERVICE_HOST, since they are both required to connect to Yurthub. I've also updated the issue title and description to reflect this. Would appreciate your reviews again, thanks!