Provision to remote k8s clusters

[mlbiam]

Goal is to be able to provision to remote k8s clusters using the same API used to provision to the local cluster. Need to be able to interact with the same APIs, building an additional API is neither desired nor required.

Option 1 - Service Account

A ServiceAccount with cluster-admin privileges is provisioned in the target cluster and stored as a Secret in the control plane cluster's OpenUnison.

Pros:

• Simple
• Users are familiar with the pattern
• Works with any cluster

Cons:

• ServiceAccount tokens not meant to be used outside of the cluster
• Tokens have no expiration
• manual rotation required
• if compromised, no way to know

Option 2 - Certificate Authentication

A key pair and CSR is generated from the control plane OpenUnison, submitted to the target cluster's CertificateSigningRequest API. The approved certificate is imported into OpenUnison and provided cluster-admin rights via RBAC. OpenUnison uses the certificate for interacting with the target cluster.

Pros:

• K8s native process
• The "secret" (Private Key), never leaves the control plane cluster

Cons:

• Many "managed" clusters don't support signing arbitrary CSRs via CertificateSigningRequest API.
• Requires direct connection to the API server
• Requires multiple steps, kind of kludgy for users that don't have experience with PKI management
• Manual rotation
• Api server doesn't recognize CRLs, so certificate can't be revoked

Option 3 - Service Proxy

Deploy an OpenUnison instance to the target cluster who's ServiceAccount has cluster-admin privileges. The OpenUnison is a reverse proxy to the target cluster's API server that expects OpenID Connect id_token in each request issued by the control plane OpenUnison. The target OpenUnison would get the control plane OpenUnison's signature validation public key via the oidc discovery API. The id_token generated by the control plane's OpenUnison would have a one minute time-to-live to limit the impact if it were compromised.

Pros:

• The "secrets" (target's Secret and control plane's private key for signing id_token) never leaves their respective cluster
• The target OpenUnison can leverage the TokenRequest api to project rotating tokens for its ServiceAccount to mitigate a compromised token
• The control plane OpenUnison can generate a new signing keypair at any time, with target cluster's OpenUnison getting that new verification key automatically via the oidc discovery API.
• Works with Ingress and multiple TLS termination points

Cons:

• Requires an OpenUnison on every target cluster (though likely will already have one for cluster authentication)
• OpenUnison on target cluster needs to be privileged - could run separately from authentication cluster to limit potential scope of impact?
• Target OpenUnison must be able to communicate with control plane OpenUnison to support oidc discovery API

Option 4 - Message bus based communication

Deploy an OpenUnison instance to the target cluster who's ServiceAccount has cluster-admin privileges. The target cluster OpenUnison is not exposed via Ingress. It listens for requests on the same message bus the control plane cluster runs on. When the control plane OpenUnison wants to communicate with a target OpenUnison it does so via message bus. The request is sent via one queue and the response to another.

Pros:

• No need to provide an Ingress for the target cluster openunison, potentially exposing a ServiceAccount to an attacker from outside of the cluster
• Can deploy target openunison independent of the cluster's login portal

Cons:

• Potentially a complex integration
• Dependent on user's message bus infrastructure
• If using integrated ActiveMQ, additional "Credentials" need to be created for each target and ActiveMQ needs to be exposed to outside clusters.
• Asynchronous communications can be complex

[mlbiam]

functional, but needs quite a bit of docs. will add in next release