Closed rambohe-ch closed 7 months ago
@gnunu @JameKeal @YTGhost PTAL
I'd like to try this. /assign @huangchenzhao
@rambohe-ch Can we automatically add the annotation ["apps.openyurt.io/exclude-host-network-pool"] = true
to pods with hostNetwork=false
?
@rambohe-ch Can we automatically add the
annotation ["apps.openyurt.io/exclude-host-network-pool"] = true
to pods withhostNetwork=false
?
What I'm wondering is, do all pods with hostNetwork=false
need to be added annotation ["apps.openyurt.io/exclude-host-network-pool"] = true
?
In that way, all the pods with hostNetwork=false
can't be scheduled to nodes in hostNetwork mode NodePool, although ports conflict doesn't exists. Is this a scheduling limitation? We could have a wider range of scheduling scope for those pods which don't have ports conflict.
Or we just add the annotation ["apps.openyurt.io/exclude-host-network-pool"] = true
mannually? but how to choose which pods should be added this annotation?
I don't figure it out yet, what do you think?
@rambohe-ch Can we automatically add the
annotation ["apps.openyurt.io/exclude-host-network-pool"] = true
to pods withhostNetwork=false
?
@YTGhost @huangchenzhao I mean that end user should add this annotation(annotation ["apps.openyurt.io/exclude-host-network-pool"] = true) to pods with hostNetwork=false manually if they don't want to run the pods on edge node in hostNetwork mode NodePool.
@rambohe-ch Can we automatically add the
annotation ["apps.openyurt.io/exclude-host-network-pool"] = true
to pods withhostNetwork=false
?What I'm wondering is, do all pods with
hostNetwork=false
need to be addedannotation ["apps.openyurt.io/exclude-host-network-pool"] = true
?In that way, all the pods with
hostNetwork=false
can't be scheduled to nodes in hostNetwork mode NodePool, although ports conflict doesn't exists. Is this a scheduling limitation? We could have a wider range of scheduling scope for those pods which don't have ports conflict.Or we just add the
annotation ["apps.openyurt.io/exclude-host-network-pool"] = true
mannually? but how to choose which pods should be added this annotation?I don't figure it out yet, what do you think?
@huangchenzhao End users should add annotation ["apps.openyurt.io/exclude-host-network-pool"] = true manually to pods if they don't want to run pods on edge nodes in hostNetwork mode nodepool.
This means if pods with hostNetwork=false has no annotation ["apps.openyurt.io/exclude-host-network-pool"] = true, these pods maybe scheduled on the edge node in hostNetwork mode NodePool, then port conflicts maybe happen and pods start failure. so end users can avoid pods start failure by adding annotation ["apps.openyurt.io/exclude-host-network-pool"] = true manually.
@rambohe-ch Can we automatically add the
annotation ["apps.openyurt.io/exclude-host-network-pool"] = true
to pods withhostNetwork=false
?What I'm wondering is, do all pods with
hostNetwork=false
need to be addedannotation ["apps.openyurt.io/exclude-host-network-pool"] = true
? In that way, all the pods withhostNetwork=false
can't be scheduled to nodes in hostNetwork mode NodePool, although ports conflict doesn't exists. Is this a scheduling limitation? We could have a wider range of scheduling scope for those pods which don't have ports conflict. Or we just add theannotation ["apps.openyurt.io/exclude-host-network-pool"] = true
mannually? but how to choose which pods should be added this annotation? I don't figure it out yet, what do you think?@huangchenzhao End users should add annotation ["apps.openyurt.io/exclude-host-network-pool"] = true manually to pods if they don't want to run pods on edge nodes in hostNetwork mode nodepool.
This means if pods with hostNetwork=false has no annotation ["apps.openyurt.io/exclude-host-network-pool"] = true, these pods maybe scheduled on the edge node in hostNetwork mode NodePool, then port conflicts maybe happen and pods start failure. so end users can avoid pods start failure by adding annotation ["apps.openyurt.io/exclude-host-network-pool"] = true manually.
understood, thanks for detailed explanation! working on that.
@huangchenzhao filter hostnetworkpropagation also need to be removed, so pods in HostNetwork mode nodepool will not be effected.
What would you like to be added:
A filter named hostnetworkpropagation in YurtHub component is used for mutating
pod.spec.HostNetwork
field to true in order to adapt hostNetwork mode NodePool. All pods on nodes in the hostNetwork mode NodePool should share the Host network namespace because cni plugin and component(like flannel) have not been installed in the hostNetwork mode NodePool.But there is a scenario that pods with hostNetwork=false which use the same ports maybe scheduled to the same node in hostNetwork mode NodePool, then the filter on the node mutate pod hostNetwork field from false to true, so ports conflict will happen and cause pods start failure. this problem is a out of expectation.
It is not a good idea to mutate pod hostNetwork field to true directly, because this maybe cause pods failure. so i think that it is a good idea to add
NodeAffinity
to pods in order to avoid pods to be scheduled on the nodes in the hostNetwork mode NodePool.Pods which don't want to be scheduled to nodes in hostNetwork mode NodePool should be specified by annotation explicitly, and the pod is
annotation["apps.openyurt.io/exclude-host-network-pool"] = true
A webhook will be added in yurt-manager component for adding NodeAffinity to pods with
annotation["apps.openyurt.io/exclude-host-network-pool"] = true
. Because all nodes in hostNetwork mode Nodepool havelabels[nodepool.openyurt.io/hostnetwork] = true
, so the NodeAffinity will be as following:others /kind feature