IAM-based Authentication for Kubernetes API

[adambom]

The Problem

We have a team of engineers responsible for maintaining our infrastructure, and these engineers will need access to our Kubernetes clusters. Right now I've been the only one accessing it, and if anyone else ever needs to access the cluster we either need to share the client certificates or generate new ones. As we scale the team, this approach isn't going to work.

In a dream world, Kubernetes would able to authenticate users using the same roles that we've defined in IAM. That would make it much easier for us to access and control who can see and modify our clusters.

It would also make it dead simple to integrate CI/CD with the cluster, without having to muck about with x509 certs.

Authentication modes in Kubernetes

Kubernetes supports a number of different authentication modes, documented here: http://kubernetes.io/docs/admin/authentication/

X509 Client Certs

Correct me if I'm wrong, but I think this is how it works right now. The API server verifies that the client's cert is signed by the same CA that generated the server's.

Static Token File

The server keeps a list of API tokens in a static file. When the client authenticates, it supplies a bearer token and the server matches it to the list of known tokens.

Static Password File

The server keeps a list of users in the format password,user,uid. When the client authenticates, it provides this information in the header and the server matches it to the list of username/password combos it know sabout.

Service Account Tokens

I still don't fully understand the function or mechanics of server accounts, but my understanding is these are primarily for headless users, not actual users. Service accounts can be created to generate bearer tokens and associate those tokens with the identity of the service account. Presumably, clients could just use those tokens for authentication.

OpenID Connect Tokens

This allows Kubernetes to authenticate users against an OAuth-like identity provider.

Webhook Token Authentication

Like static token file, but the server authenticates bearer tokens against a server over a webhook rather than a local file.

Keystone

I don't claim to know anything about this

The Proposal

Use webhooks to authenticate users against IAM. When the client connects, it should supply a bearer token of the form USERNAME:AWS_ACCESS_KEY_ID:AWS_SECRET_KEY. The API server will be configured to forward this request to a service running inside a container on the master node, which will call out to IAM. This request will look like this:

{
  "apiVersion": "authentication.k8s.io/v1beta1",
  "kind": "TokenReview",
  "spec": {
    "token": "USERNAME:AWS_ACCESS_KEY_ID:AWS_SECRET_KEY"
  }
}

The proxy service will run the equivalent of:

AWS_ACCESS_KEY_ID=AWS_ACCESS_KEY_ID AWS_SECRET_KEY=AWS_SECRET_KEY \
aws iam get-user USERNAME

And will return the following response for authentic users:

{
  "apiVersion": "authentication.k8s.io/v1beta1",
  "kind": "TokenReview",
  "status": {
    "authenticated": true,
    "user": {
      "username": "USERNAME",
      "uid": "42",
      "groups": [
        "developers",
        "qa"
      ],
      "extra": {
        "extrafield1": [
          "extravalue1",
          "extravalue2"
        ]
      }
    }
  }
}

We should also provide the option for the cluster administrator to override the default IAM proxy, so they can call out to their own identity providers if need be. Finally, Tack should bootstrap a superadmin IAM user at launch time.

[adambom]

@hughyoh @moore

[yagonobre]

Hi, its look nice. I was thinking to use OpenID Connect with dex and freeipa in my enviroment, what you think about?

cc: @guiocavalcanti

[adambom]

@yagonobre I think that looks interesting as well. I like that dex is capable of using one of their pluggable connectors. For example, we could benefit from tying in to our LDAP system.

The downside is that there isn't a great way of hooking in to IAM, as far as I know, but maybe that shouldn't be considered a hard requirement.

I could see a scenario where Tack ships with dex, backed by a "local" connector by default, with the option to override that to point to any of the supported authentication providers.

What are your thoughts? I'd like to hear opinions from others as well.

[yagonobre]

I plan to use RBAC for IAM. But the webhook aprouch looks interesting. Maybe we can do 2 webhooks, one for authentication and another for authorization. I can help you, what you think we write it in go?

For reference: https://www.youtube.com/watch?v=i75ysFcvCkk&list=PLlh6TqkU8kg_3FpXLlHMnoVqKZysIzXlK&index=32

[adambom]

@yagonobre Thanks for sharing that video. Very informative. I'm growing more convinced that Dex seems like a good approach. Could you share a bit more about what your plans are RE: dex and freeipa?

[yagonobre]

I'm plaining to use freeipa as ldap provide, and bind groups in freeipa with my clusters.