loft-sh / vcluster-plugins

The plugin repository for vcluster. Extend virtual Kubernetes clusters with plugins.
11 stars 6 forks source link

Create a plugin for easier use of EKS SA roles #6

Closed matskiv closed 1 year ago

matskiv commented 2 years ago

What: As a vcluster user, I would like to use the "IAM roles for service accounts" feature of my host EKS cluster in a secure and convenient way.

Why: Current solutions are either cumbersome to use, or they are not as secure. One option now is to use --service-account=xyz syncer flag, create the "xyz" SA in the host cluster (with the correct annotation) and add it to was policy. This is not very secure because all pods in vcluster will have access to the sensitive env vars (e.g. AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN). Alternative option is to enable service account sync, but then the name of the synced SA is pretty cryptic, and harder to predict if the parts of the name are long and it needs to use a hash to stay under 63 characters. And this name needs to be set in the AWS policy, which might be managed by GitOps, which makes it potentially even more difficult.

How: Option A: Create a generic plugin for service account name translation mapping. It would enable service account sync, and use hooks to change how the synced SA is named based on user configuration. E.g.: this bit from plugin configuration: default/aws-privileged:default-aws-privileged would mean that when syncing default/aws-priviledge SA from vcluster to host, it should be named default-aws-privileged instead of aws-privileged-x-default-x-vclustername

Option B: Create a more tailored plugin, which would sync only the SAs that have been added to plugin configuration and have the EKS annotation. And all other service accounts would be skipped, and the usual SA (from the syncers --service-account flag) would be used. This would replace the service account syncer and hook into the pod syncer.