pulumi / pulumi-eks

A Pulumi component for easily creating and managing an Amazon EKS Cluster
https://www.pulumi.com/registry/packages/eks/
Apache License 2.0
171 stars 81 forks source link

Cluster.createOidcProvider in china regions (cn-north-1, cn-northwest-1) #534

Closed Iced-Sun closed 1 year ago

Iced-Sun commented 3 years ago

The line of https://github.com/pulumi/pulumi-eks/blob/f7ef1c09cc34a2bc2914b49fab46eafa8135896f/nodejs/eks/cluster.ts#L759 hard-codes the endpoints of EKS as eks.*.amzonaws.com.

Per https://docs.amazonaws.cn/en_us/aws/latest/userguide/endpoints-Beijing.html and https://docs.amazonaws.cn/en_us/aws/latest/userguide/endpoints-Ningxia.html, the China regions have the EKS endpoint as eks.*.amazonaws.com.cn. A direct consequence is that currently pulumi will throw an exception of Cannot retrieve the certificate fingerprint at the issuer URL: https://oidc.eks.cn-northwest-1.amazonaws.com when creating an odicProvider.

A similar case is reported in #386. I managed to work around the wrong managed policy arns and service endpoints by import { ServiceRole } from '@pulumi/eks/servicerole' and then creating the service roles with the corrected strings.

Hope the AWS region partition problem (aws-global/aws-cn/aws-us-gov) could be addressed.

arunsisodiya commented 2 years ago

@Iced-Sun - Can you please provide a sample code for fixing this issue using the service role?

I need to do that for the China region.

Iced-Sun commented 2 years ago

@Iced-Sun - Can you please provide a sample code for fixing this issue using the service role?

I need to do that for the China region.

With pleasure.

Here is everything (hopefully) that involves the aws-cn partition.

The IAM role for EKS cluster:

const cluster_role = new aws.iam.Role(`${resource_name}.AmazonEKSClusterRole`, {
    assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal({ Service: 'eks.amazonaws.com' }),
    description: 'Allows access to other AWS service resources that are required to operate clusters managed by EKS.',
    managedPolicyArns: [ 'arn:aws-cn:iam::aws:policy/AmazonEKSClusterPolicy' ]
});

The IAM role for EKS worknode:

const node_role = new aws.iam.Role(`${resource_name}.AmazonEKSNodeRole`, {
    assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal({ Service: 'ec2.amazonaws.com.cn' }),
    description: 'Allows Amazon EKS worker nodes to connect to Amazon EKS Clusters; provides read-only access to Amazon EC2 Container Registry repositories.',
    managedPolicyArns: [
        'arn:aws-cn:iam::aws:policy/AmazonEKSWorkerNodePolicy',
        'arn:aws-cn:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly'
    ]
});

The IAM OIDC provider of the EKS cluster:

const oidc_provider = new aws.iam.OpenIdConnectProvider(`${resource_name}.oidc-provider`, {
    clientIdLists: [ 'sts.amazonaws.com' ],
        // url is resolved as 'oidc.eks.cn-northwest-1.amazonaws.com.cn/id/<redacted>'
    url: cluster.identities[0].oidcs[0].issuer,
    thumbprintLists: [ '<redacted>' ]
});

The IAM role for CNI plugin:

const cni_role = new aws.iam.Role(`${resource_name}.AmazonEKSCNIRole`, {
    assumeRolePolicy: {
        Version: '2012-10-17',
        Statement: [{
            Effect: 'Allow',
                        // oidc_provider.arn is resolved as 'arn:aws-cn:iam::<redacted>:oidc-provider/oidc.eks.cn-northwest-1.amazonaws.com.cn/id/<redacted>' 
            Principal: { Federated: oidc_provider.arn },
            Action: 'sts:AssumeRoleWithWebIdentity',
            Condition: {
                StringEquals: {
                                        // oidc_provider.url is resolved as 'oidc.eks.cn-northwest-1.amazonaws.com.cn/id/<redacted>'
                    [ `${oidc_provider.url}:sub` ]: [ 'system:serviceaccount:kube-system:aws-node' ]
                }
            }
        }]
    },
    managedPolicyArns: [ 'arn:aws-cn:iam::aws:policy/AmazonEKS_CNI_Policy' ]
});

Those should be the minimal prerequisties to provision an EKS cluster in the regions of cn-north-1/cn-northwest-1.

arunsisodiya commented 2 years ago

@Iced-Sun - Thanks for the response. :) I will try this. By the way, what is the value for resource_name?

Iced-Sun commented 2 years ago

@Iced-Sun - Thanks for the response. :) I will try this. By the way, what is the value for resource_name?

That should not be relevant. For reference, it is set to rnd.

Thanks!

arunsisodiya commented 2 years ago

Thanks. And what about linking those roles in the cluster? How can we do that?

Iced-Sun commented 2 years ago

Thanks. And what about linking those roles in the cluster? How can we do that?

  1. cluster role: c.f. https://www.pulumi.com/registry/packages/eks/api-docs/cluster/#servicerole_nodejs
  2. node role: c.f. https://www.pulumi.com/registry/packages/eks/api-docs/managednodegroup/#noderole_nodejs