drone / autoscaler

Automatically adds or removes instances based on build volume
https://autoscale.drone.io
Other
193 stars 89 forks source link

Update amazon session.New to support web identity token from service account in K8s #134

Open Lowess opened 1 year ago

Lowess commented 1 year ago

Hi Everyone !

While trying to get the drone autoscaler running on Amazon EKS Kubernetes cluster I encountered an issue with the way the AWS SDK handles the session when using an IAM role.

The drone autoscaler deployment is assigned a ServiceAccount to grant the service access to an IAM role (IRSA / OIDC IAM role dedicated to the pod) as stated in this documentation https://docs.aws.amazon.com/eks/latest/userguide/associate-service-account-role.html

While digging into AWS SDK issues I found this relevant issue that makes a mention of using session.NewSession instead of session.New which does not handle well web identity token files https://github.com/aws/aws-sdk-go/issues/4436

I made a local build of the project and took it for a spin and it works like charm now. The Drone autoscaler service is able to assume the ServiceAccount IAM role instead of the IAM role assigned to the EC2 machine.

AWS_IAM: true
DRONE_AMAZON_IAM_PROFILE_ARN: arn:aws:iam::123456789101:instance-profile/drone-autoscaler
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::123456789101:role/drone-autoscaler
    meta.helm.sh/release-name: drone-autoscaler--production
    meta.helm.sh/release-namespace: cicd
  creationTimestamp: "2023-04-18T09:55:18Z"
  labels:
    release: drone-autoscaler--production
  name: drone-autoscaler--production-monochart-default
  namespace: cicd
secrets:
- name: drone-autoscaler--production-monochart-default-token-fzgrm
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::123456789101:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/<OIDC_ID>"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringLike": {
                    "oidc.eks.us-east-1.amazonaws.com/id/<OIDC_ID>": "system:serviceaccount:cicd:drone-autoscaler--production-monochart-default"
                }
            }
        }
    ]
}
Lowess commented 1 year ago

@julienduchesne do you mind checking if this sounds legitimate ? That would be great if this could go through a future release.

Thanks !

julienduchesne commented 1 year ago

@julienduchesne do you mind checking if this sounds legitimate ? That would be great if this could go through a future release.

Thanks !

I’m not a Drone maintainer, just a random guy launching agents in AWS from a k8s cluster in GCP. So about auth, I have no idea

Lowess commented 3 months ago

@vistaarjuneja any changes you could review this PR and get it merged ? I saw you recently merged changes related to the Amazon provider.

Many thanks 🙏