Teleport on EKS guide relies on ec2 nodes ambiant credentials

[hugoShaka]

Expected behavior:

Teleport, as a security product, should not advise users to grant Dynamo, S3 and Route53 read-write permissions to all workload running in the EKS cluster.

Current behavior:

Teleport on EKS guide advises user to do something that can be exploited if they run other workload in the cluster.

This is not only a documentation issue, there are important technical and UX considerations:

• This technical choice was made because setting up aws-iam-controller and IAM for impersonation can be hard and complex (try to manually follow their docs without a strong Kubernetes and IAM experience)
• We want to do the right thing (the secure one) while keeping the happy path as simple as possible (and today it's already quite complex)
• Increasing the setup complexity will increase setup time, mistakes, reduce successful deployments and increase time to value
• Advising the right thing will avoid solution engineers to have to explain correct practices to customers, which will help reduce onboarding time
• Teleport is not handling properly AWS credentials, which makes passing AWS credentials harder (see technical context)
• We don't want to cover how to install Kubernetes and have to support a diverse range of setup scenarios, including a user brave enough to create an EKS cluster via the web console

Technical context:

The only approach to do this more securely is to use an AWS Service Account. There are two common ways of using an AWS SA in Kubernetes:

• Pass a SA Key through a file or environment variable (blocked by #2781)
• Pass short-lived tokens on pod's execution via aws-iam-controller.

The latter is the most secure way to do it, but adds up to 8 additional steps in the deployment guide.

[hugoShaka]

Possible approaches:

• require users to setup aws-iam-controller at the beginning of the guide -> we might loose a lot of users here
• advice users to pass a key through env vars -> less secure but could be coupled with the aws-iam-controller suggestion
• switch to a more opiniated "teleport on EKS" guide and choose the cluster setup tool: eksctl, terraform, ansible, cloudformation -> yet another installation way to maintain (and we have not been really good so far at maintaining those), can drastically reduce time to value at the cost of applying to less customers (we can suppose more power users with less standard setups already have the IAM controler running)

[hugoShaka]

Note: we have similar scenarios with other cloud providers (azure and GCP) the default approach has been to rely on keys. It kinda makes sense for azure db discovery, there is an open issue to support and document workload identity for GKE.

[ptgott]

If we're going to elevate the approach laid out in the EKS/Helm guide to be the one we recommend to get most users to production, we should make this issue one of our top priorities for the next quarter.

[ptgott]

@alexfornuto @xinding33 I meant to tag you about the comment above but forgot

[hugoShaka]

Update: in EKS, aws now runs the iam controller for us. This makes the setup a bit easier