Create shared package for watching Nodes in remote Clusters

JoelSpeed commented 4 years ago

Note: this is only applicable once #2250 is merged

⚠️ Cluster API maintainers can ask to turn an issue-proposal into a CAEP when necessary, this is to be expected for large changes that impact multiple components, breaking changes, or new large features.

Goals

Make it easy for controller developers within the CAPI space to react to events on Node objects in target clusters
Provide a common way of watching Node objects within remote clusters

Non-Goals/Future Work

Implement remote watching of Nodes in all controllers

User Story

As a developer I would like the controllers I am writing to be able to react to events on Nodes in remote clusters so that my controller can reconcile the status of objects or take action based on the state of the Nodes

Detailed Description

The Machine Health Checking controller implemented primarily in #2250 needed to watch Nodes in remote clusters to be able to react as quickly as possible to Nodes which might be going unhealthy.

To achieve this, a method was implemented that keeps a map of clusters to informers that allows the controller to add event handlers for events coming from nodes in remote clusters.

This could be moved into the external package and made more reusable by making it a method on a new struct which keeps the map, lock and controller as fields

type RemoteClusterWatcher struct {
  controller               controller.Controller
  clusterNodeInformers.    map[types.NamespacedName]cache.Informer
  clusterNodeInformersLock *sync.Mutex
}

And changing the signature of the WatchClusterNodes method to allow the event handler to be passed.

It is currently assumed that each controller would only want to register one event handler per cluster, I'm not sure how true that is and as such, we should try to add the ability to check if event handler functions have been registered and add more if required.

/kind proposal