kube-aws / kube-spot-termination-notice-handler

A Kubernetes DaemonSet to gracefully delete pods 2 minutes before an EC2 Spot Instance gets terminated
Apache License 2.0
378 stars 77 forks source link

[Question] upgrading aws-cli? Supporting serviceAccountAnnotations? #43

Open ngkuznetsov opened 4 years ago

ngkuznetsov commented 4 years ago

Hi,

We would like to use the detaching feature.

  1. We use kubectl annotate serviceaccount to provide a service-account for spot-termination-handler pod.

  2. It provides AWS_ROLE_ARN & AWS_WEB_IDENTITY_TOKEN_FILE environment variables.

    $ env | grep AWS
    AWS_ROLE_ARN=arn:aws:iam::XXXXXXXXXX:role/XXXXXXXXXXXX-eu-west-1-kube-system-spot-termination-handler
    AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token
  3. AWS cli version 1.16.199 installed in the docker image kubeaws/kube-spot-termination-notice-handler:1.13.7-1 does not support resolving credentials via STS AssumeRoleWithWebIdentity. There is an error:

    An error occurred (AccessDenied) when calling the DescribeAutoScalingInstances operation: User: arn:aws:sts::XXXXXXXXXXX:assumed-role/XXXXXX-eks-worker-eu-west-1/i-xxxxxxxx is not authorized to perform: autoscaling:DescribeAutoScalingInstances

AWS cli ignores AWS_ROLE_ARN & AWS_WEB_IDENTITY_TOKEN_FILE. This feature was introduced only in the version 1.16.210 - https://github.com/aws/aws-cli/blob/develop/CHANGELOG.rst#116210

Could you please upgrade the aws cli (with version >1.16.210) and build a new docker image.


Additionally, it would be great to have an option to add rbac.serviceAccountAnnotations to the Helm Chart as was done for cluster-autoscaler, for example: https://github.com/helm/charts/blob/master/stable/cluster-autoscaler/templates/serviceaccount.yaml#L10

In this case we can replace running kubectl annotate serviceaccount and restarting pods manually with setting annotations as Helm values:

--set rbac.serviceAccountAnnotations."eks\.amazonaws\.com/role-arn"=${TF_STATE[cluster_autoscaler_iam_role]} \

Thank you.

Best regards, Mikalai