[EKS] [request]: Disable anonymous-auth to kube-apiserver

[jessebye]

Tell us about your request Add the ability to pass the --anonymous-auth=false flag to the kube-apiserver. Passing in other arbitrary flags to the apiserver would also be nice, but not as necessary.

Which service(s) is this request for? EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? Several attacks target kube-apiserver through anonymous auth. Billion laughs is a good example of a very recent attack that could be exploited by anonymous auth.

Disabling anonymous auth completely is a security best-practice and mitigates a whole class of attacks.

Are you currently working around this issue? By keeping our clusters up-to-date with security fixes and protecting the endpoint URLs from public discovery. We could use private API endpoints, but this is complicated due to https://github.com/aws/containers-roadmap/issues/221

Additional context n/a

Attachments n/a

[szymonpk]

Can you even run any anonymous actions with EKS? I always believed IAM identity is mandatory when interacting with EKS clusters - https://docs.aws.amazon.com/en_pv/eks/latest/userguide/managing-auth.html.

[jessebye]

@szymonpk You can run anonymous actions with EKS. It is possible to completely bypass IAM for user authentication if you wish, it's just common to use IAM for this on EKS.

Here's a test I did demonstrating that Anonymous auth isn't completely disabled:

The ~/.kube/config looks like this:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: [redacted]
    server: https://[redacted].[redacted].us-east-2.eks.amazonaws.com
  name: arn:aws:eks:us-east-2:[redacted]:cluster/test
contexts:
- context:
    cluster: arn:aws:eks:us-east-2:[redacted]:cluster/test
    user: arn:aws:eks:us-east-2:[redacted]:cluster/test
  name: dev
current-context: dev
kind: Config
preferences: {}

Then running a command, I get a prompt for username/password and enter invalid info (asdf):

kubectl cluster-info dump
Please enter Username: asdf
Please enter Password: Error from server (Forbidden): nodes is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope

As the error message indicates, I am authenticated as an anonymous user, but get forbidden because the anonymous user doesn't have authorization for the action.

[stefansedich]

@szymonpk can also hit the API directly in this case to list APIs or hit the health check unauthenticated, I simply turned off public access to our API and access it through a VPN but it would be nice to set --anonymous-auth=false too.

curl -v -k https://EKS_API_HOST/healthz
curl -v -k https://EKS_API_HOST/apis/

[0x91]

Since the GuardDuty EKS integration was released we are getting notifications about scrapers hitting our EKS control plane APIs. Being able to disable anonymous auth would be great.

[ghostsquad]

~2 years later. We really need this.

[code4happylife]

This is for the security, which however many clusters were not set this parameter by default.

[kahirokunn]

I need this.

[joebowbeer]

This is securely configured to be false in the optimized AMI (and bottlerocket, too, I hope)

https://github.com/awslabs/amazon-eks-ami/blob/master/files/kubelet-config.json#L7

[jessebye]

@joebowbeer That's great, but is mitigating a separate issue (anonymous access to kubelet on nodes). This issue is different, as it involves anonymous access to kube-apiserver. Since the kube-apiserver is not running on nodes exposed to us (EKS abstracts the API control plane nodes away from us), we cannot pass a flag to disable anonymous auth there.

[RamazanKara]

We are also looking for a solution for this.

[dims]

related https://raesene.github.io/blog/2023/03/18/lets-talk-about-anonymous-access-to-Kubernetes/

[joebowbeer]

related https://raesene.github.io/blog/2023/03/18/lets-talk-about-anonymous-access-to-Kubernetes/

This recent article claims that all major distributions of Kubernetes enable this by default.

Because, perhaps, quoting from CIS Benchmark:

If you are using RBAC authorization, it is generally considered reasonable to allow anonymous access to the API Server for health checks and discovery purposes, and hence this recommendation is not scored.

[raesene]

So a bit more context might be useful here (I'll update that post as it's proving of interest :) )

From a security perspective there are two risks that I can see from allowing anonymous authentication to a Kubernetes cluster. The first is that there is information disclosure of things like version number which is useful to attackers in targeting exploits. This is because of the system:public-info-viewer cluster role/cluster role binding which provides access to a set of endpoints for the system:unauthenticated group.

Then there's the more serious (but less likely) risk which is that a cluster operator who makes an RBAC mistake (or is duped into making the mistake) of binding additional rights to the system:anonymous user or system:unauthenticated group will allow anyone on the Internet to use those rights without authentication.

Whilst I understand that there have been reasons why unauthenticated access to k8s clusters has been seen as necessary (which is why I wrote that bit in the CIS benchmark), in terms of applying defence-in-depth to cluster configuration, I would generally recommend disabling anonymous auth to the Kubernetes cluster if possible (as well as removing the API server from the Internet, but that's another story :) ).

[joebowbeer]

@raesene writes:

Whilst I understand that there have been reasons why unauthenticated access to k8s clusters has been seen as necessary (which is why I wrote that bit in the CIS benchmark), in terms of applying defence-in-depth to cluster configuration, I would generally recommend disabling anonymous auth to the Kubernetes cluster if possible (as well as removing the API server from the Internet, but that's another story :) ).

To be clear, have you changed your mind about the scoring (aka automation) of this recommendation?

IIUC, you wish there were a practical way (this issue) to disable anonymous access in EKS (and all distributions), but until there is, the rule should not be scored because, in addition to there being no practical fix (yet), the threat can be mitigated by using RBAC and removing the public endpoint.

[raesene]

So from a CIS benchmark perspective automation/scoring is slightly academic in relation to EKS as there's an EKS specific benchmark which doesn't mention this setting, as it's not available for cluster operators to change :)

FWIW I probably would change this one now, as the threat model/environment has moved on a long way since it was drafted. that change came in for the k8s 1.4 benchmark which was Kubernetes 1.13 and k8s security has improved a lot since then!

From a security standpoint, it is not ideal to have anonymous access allowed, for the reasonse I mentioned, so from a purely security standpoint, it should be disabled.

IIRC (and this was some years ago now) the reason that one went in with the text about RBAC is that (at the time) there were concerns from the Kubernetes project about the downsides of disabling it, the reality is that some clusters will likely want the access and others will not, so ideally it would be exposed as a setting either at cluster creation, or at runtime, if possible.

On the topic of using RBAC to mitigate the risks, it's worth noting there may be some complications there with things like removing the default bindings (https://kubernetes.io/docs/reference/access-authn-authz/rbac/#default-roles-and-role-bindings) , as the API server may re-create RBAC rules from the default set when it's re-started.

[Rvarman0110]

Hey just a doubt so I am using a managed cluster from alicloud and when I am trying to curl

https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}

I am not passing any credentials ( when I am trying this on Azure I am getting an 401 error)

But now I am getting 403 { "kind": "Status", "apiVersion": "v1", "metadata": {}, "status": "Failure", "message": "forbidden: User \"system:anonymous\" cannot get path \"/\"", "reason": "Forbidden", "details": {}, "code": 403

• Connection #0 t }

I guess it is because of the anonymous flag which might be set as true

For ACK how can i set it as false , also I raised a ticket on alibaba to set this as false in kubelet conf file but I didn't get any reply on that , If you guys have any thoughts It would be really helpful for me

[Rvarman0110]

@raesene I have a question for you As you said, From a security perspective there are two risks that I can see from allowing anonymous authentication to a Kubernetes cluster. The first is that there is information disclosure of things like version number which is useful to attackers in targeting exploits. This is because of the system:public-info-viewer cluster role/cluster role binding which provides access to a set of endpoints for the system:unauthenticated group.

Now for my managed cluster which is on ACK can I try deleting this clusterroles and bindings ?? Is it a good way to follow ?

[raesene]

@Rvarman0110 so you could delete cluster roles and bindings, but there are two potential problems that I can think of, and possibly others I've not thought of :D

1. There's something in your cluster that relies on those rights (e.g. health checks), which breaks when you delete them.
2. As mentioned up the thread, default rights may be restored on API server re-start https://kubernetes.io/docs/reference/access-authn-authz/rbac/#default-roles-and-role-bindings , so your changes may be reverted.