Proposal: Colocate control plane components by default

ironcladlou commented 3 years ago

We anticipate the desire for colocation of control plane components to limit the impact of cross-node issues, optimize the communications paths between components, etc (see https://github.com/openshift/hypershift/issues/235).

This proposal is to define colocation in terms of pod affinity rules based on the kubernetes.io/hostname topology key and to set default affinity rules for control plane component pods like:

podAffinity:
  preferredDuringSchedulingIgnoredDuringExecution:
    - weight: 100
      podAffinityTerm:
        labelSelector:
          matchExpressions:
            - key: hypershift.openshift.com/control-plane
              operator: In
              values: ["$cluster-id"]
        topologyKey: "kubernetes.io/hostname"

This default would necessarily preclude the ability of users to spread individual components across nodes, but we have no known use case for that. This default would eliminate the need for any direct affinity configuration API at this time barring some new requirements.

TODO: Decide on an appropriate key value (e.g. the hostedcluster UUID, or namespace+name tuple, or...) TODO: Define the exhaustive list of control plane components which to which these defaults will be applied.

relyt0925 commented 3 years ago

I am in favor of namespace+name tuple over uuid

relyt0925 commented 3 years ago

Everything scheduled in the operations of a master should also have this label. See this comment for that list: https://github.com/openshift/hypershift/issues/371#issuecomment-888441138

relyt0925 commented 3 years ago

/assign @csrwng

ironcladlou commented 3 years ago

Tracked in https://issues.redhat.com/browse/HOSTEDCP-194

openshift / hypershift

Proposal: Colocate control plane components by default #370