search

pulumi / pulumi-eks

A Pulumi component for easily creating and managing an Amazon EKS Cluster
https://www.pulumi.com/registry/packages/eks/
Apache License 2.0
169 stars 78 forks source link

NodeGroupV2 cannot be created using eks.Cluster from stack #855

Open klis opened 1 year ago

klis commented 1 year ago

What happened?

In the pulumi-eks version v0.42.7, I could use eks.Cluster from another stack via pulumi.StackReference. In version v1.0.0 there is no longer the case. I'm pretty sure that PR #815 broke that possibility, since cluster.core is determined using instanceof operator

const core = args.cluster instanceof Cluster ? args.cluster.core : args.cluster;

In the older version, cluster.core was determined with a function and a simple if statement.

function isCoreData(arg: NodeGroupOptionsCluster): arg is CoreData {
    return (arg as CoreData).cluster !== undefined;
}

I was wondering if that is on purpose because when I retrieve eks.Cluster from the stack, it will not be a valid instance of the object since it is not a Pulumi object.

Expected Behavior

eks.Cluster can be fetched from the Stack and NodeGroupV2 can be created using this cluster as a reference.

Steps to reproduce

Create a eks.Cluster in one stack Sample code:

const eksClusterRole = new aws.iam.Role('role', {
    assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal({
        Service: "ec2.amazonaws.com",
    }),
}, {
    parent: this
})

new aws.iam.RolePolicyAttachment('eksClusterRoleAttachment-0', {
    policyArn: "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy",
    role: eksClusterRole
}, {
    parent: eksClusterRole
})

new aws.iam.RolePolicyAttachment('eksClusterRoleAttachment-1', {
    policyArn: "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy",
    role: eksClusterRole
}, {
    parent: eksClusterRole
})

new aws.iam.RolePolicyAttachment('eksClusterRoleAttachment-2', {
    policyArn: aws.iam.ManagedPolicy.AmazonEC2ContainerRegistryReadOnly,
    role: eksClusterRole
}, {
    parent: eksClusterRole
})

new aws.iam.RolePolicyAttachment('eksClusterRoleAttachment-3', {
    policyArn: "arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy",
    role: eksClusterRole
}, {
    parent: eksClusterRole
})

export const instanceProfile = new aws.iam.InstanceProfile('instanceProfile', {
    role: eksClusterRole
}, {
    parent: this
})

export const cluster = new eks.Cluster('test', {
    createOidcProvider: true,
    desiredCapacity: 2,
    instanceRoles: [
        eksClusterRole
    ],
    instanceType: "m5ad.xlarge",
    maxSize: 5,
    minSize: 1,
    nodeAssociatePublicIpAddress: false,
    nodeRootVolumeSize: 100,
    nodeUserData: nodeUserData,
    privateSubnetIds: subnetIds,
    skipDefaultNodeGroup: true,
    version: '1.24',
    vpcId: vpcId,
});

Then in another stack, try to create a NodeGroupV2 and pass the eks.Cluster from the stack reference

const stackReference = new pulumi.StackReference(stackReferenceName)

const cluster = (stackReference.requireOutput("cluster") as unknown) as eks.Cluster
const instanceProfile = (stackReference.requireOutput("instanceProfile") as unknown) as aws.iam.InstanceProfile

pulumi.output(cluster).apply(unwrapped => {

// code is checking for things that are not in the eks.Cluster from stack
// we are relying that this already exists since we are using existing EKS cluster
unwrapped.core.eksNodeAccess = undefined
unwrapped.core.vpcCni = undefined

const node = new eks.NodeGroupV2('node01', {
    amiId: 'ami-06bf8e441ff8de6c6',
    cluster: unwrapped,
    instanceProfile: instanceProfile,
    instanceType: 'm5ad.2xlarge',
    desiredCapacity: 1,
    maxSize: 2,
    minSize: 1,
    nodeAssociatePublicIpAddress: false,
    nodeRootVolumeSize: 50,
    nodeRootVolumeType: 'gp3',
    nodeSubnetIds: subnetIds,
})

Run pulumi up --stack ng-test and you get an error

Previewing update (amazon.test):
     Type                                          Name                                                  Plan       Info
     pulumi:pulumi:Stack                           portal-provisioner-amazon.test                                   2 errors
 +   └─ reversinglabs:swa-saas:PortalEKSNodeGroup  amazon-nodegroup01                                    create
 +      ├─ tls:index:PrivateKey                    amazon-test-node-ssh-key                              create
 +      ├─ aws:ec2:KeyPair                         amazon-test-keypair                                   create
 +      └─ eks:index:NodeGroupV2                   amazon-test-node01                                    create
 +         ├─ aws:ec2:SecurityGroup                amazon-test-node01-nodeSecurityGroup                  create
 +         ├─ aws:ec2:SecurityGroupRule            amazon-test-node01-eksNodeClusterIngressRule          create
 +         ├─ aws:ec2:SecurityGroupRule            amazon-test-node01-eksNodeInternetEgressRule          create
 +         ├─ aws:ec2:SecurityGroupRule            amazon-test-node01-eksNodeIngressRule                 create
 +         ├─ aws:ec2:SecurityGroupRule            amazon-test-node-ssh                                  create
 +         ├─ aws:ec2:SecurityGroupRule            amazon-test-node01-eksExtApiServerClusterIngressRule  create
 +         └─ aws:ec2:SecurityGroupRule            amazon-test-node01-eksClusterIngressRule              create

Diagnostics:
  pulumi:pulumi:Stack (portal-provisioner-amazon.test):
    error: TypeError: Cannot read properties of undefined (reading 'nodeSecurityGroup')
        at /mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/nodegroup.ts:953:32
        at /mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/output.ts:383:31
        at Generator.next (<anonymous>)
        at /mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/pulumi/output.js:21:71
        at new Promise (<anonymous>)
        at __awaiter (/mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/pulumi/output.js:17:12)
        at applyHelperAsync (/mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/pulumi/output.js:236:12)
        at /mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/output.ts:302:65
        at processTicksAndRejections (node:internal/process/task_queues:96:5)
    error: TypeError: Cannot read properties of undefined (reading 'data')
        at /mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/nodegroup.ts:1123:95
        at /mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/output.ts:383:31
        at Generator.next (<anonymous>)
        at /mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/pulumi/output.js:21:71
        at new Promise (<anonymous>)
        at __awaiter (/mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/pulumi/output.js:17:12)
        at applyHelperAsync (/mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/pulumi/output.js:236:12)
        at /mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/output.ts:302:65
        at processTicksAndRejections (node:internal/process/task_queues:96:5)

Output of pulumi about

CLI Version 3.46.1 Go Version go1.19.2 Go Compiler gc

Plugins NAME VERSION aws 5.28.0 aws 5.16.2 awsx 1.0.1 docker 3.6.1 eks 1.0.1 flux 0.22.2 github 5.3.0 kubernetes 3.23.1 nodejs unknown postgresql 3.6.0 random 4.10.0 tls 4.8.0

Host OS debian Version 10.13 Arch x86_64

This project is written in nodejs: executable='/usr/bin/node' version='v17.9.0'

Backend Name PC-TKLISANIC2 URL file://~ User tklisanic Organizations

Dependencies: NAME VERSION @pulumi/awsx 1.0.1 @pulumi/kubernetesx 0.1.6 @types/js-yaml 4.0.5 @types/node 16.18.11 @pulumi/aws 5.28.0 @pulumi/github 5.3.0 @pulumi/postgresql 3.6.0 @pulumi/pulumi 3.53.1 @pulumi/random 4.10.0 @pulumi/tls 4.8.0 @worawat/flux 0.22.2 yaml 2.2.1 @pulumi/eks 1.0.1

Pulumi locates its logs in /tmp by default

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

squaremo commented 1 year ago

Thanks for posting this, and for digging into the code yourself. I think it's a reasonable expectation, to be able to supply an object from StackReference.getOutput(...) as an input expecting that kind of object. I don't know the code here well, but this sticks out to me:

const cluster = (stackReference.requireOutput("cluster") as unknown) as eks.Cluster

Should you cast to Output<eks.Cluster> here?

klis commented 1 year ago

Thanks for posting this, and for digging into the code yourself. I think it's a reasonable expectation, to be able to supply an object from StackReference.getOutput(...) as an input expecting that kind of object. I don't know the code here well, but this sticks out to me:

const cluster = (stackReference.requireOutput("cluster") as unknown) as eks.Cluster

Should you cast to Output<eks.Cluster> here?

I need to test this. Hopefully, tomorrow I can try this on.

klis commented 1 year ago

@squaremo still getting the same error. When I evaluate args.cluster instanceof Cluster it is still false.

    error: TypeError: Cannot read properties of undefined (reading 'nodeSecurityGroup')
        at /mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/nodegroup.ts:953:32
        at /mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/output.ts:383:31
        at Generator.next (<anonymous>)
        at /mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/pulumi/output.js:21:71
        at new Promise (<anonymous>)
        at __awaiter (/mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/pulumi/output.js:17:12)
        at applyHelperAsync (/mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/pulumi/output.js:236:12)
        at /mnt/c/git/github.rl.lan/tklisanic/infrastructure/portal-provisioning/portal-provisioner/node_modules/@pulumi/output.ts:302:65
        at processTicksAndRejections (node:internal/process/task_queues:96:5)
pierskarsenbarg commented 1 year ago

@klis I've not been able to replicate this even with v0.42.7. Can you add your code to a GH repo that I can clone and run pulumi up against on my laptop?

klis commented 1 year ago

@klis I've not been able to replicate this even with v0.42.7. Can you add your code to a GH repo that I can clone and run pulumi up against on my laptop?

Sorry, I have this in our company's proprietary code. But I will prepare something and link it here so you can test it out.

klis commented 11 months ago

@klis I've not been able to replicate this even with v0.42.7. Can you add your code to a GH repo that I can clone and run pulumi up against on my laptop?

@pierskarsenbarg finally found the time to create a sample project. You can find it here. If you have any questions, please ping me.

gdmello commented 10 months ago

seeing this issue on current version of Pulumi-

CLI          
Version      3.78.1
Go Version   go1.20.7
Go Compiler  gc

Plugins
NAME        VERSION
aws         5.42.0
eks         1.0.3
kubernetes  3.30.2
python      unknown

Host     
OS       ubuntu
Version  22.04
Arch     x86_64

This project is written in python: executable='/home/user.name/.pyenv/shims/python3' version='3.10.12
'

Current Stack: organization/kubernetes-worker-nodes/staging

TYPE                                     URN
pulumi:pulumi:Stack                      urn:pulumi:staging::kubernetes-worker-nodes::pulumi:pulumi:Stack::kubernetes-worker-nodes-staging
pulumi:providers:pulumi                  urn:pulumi:staging::kubernetes-worker-nodes::pulumi:providers:pulumi::default
pulumi:pulumi:StackReference             urn:pulumi:staging::kubernetes-worker-nodes::pulumi:pulumi:StackReference::organization/vpc/staging
pulumi:pulumi:StackReference             urn:pulumi:staging::kubernetes-worker-nodes::pulumi:pulumi:StackReference::organization/kubernetes-control-plane/staging
pulumi:providers:aws                     urn:pulumi:staging::kubernetes-worker-nodes::pulumi:providers:aws::default_5_42_0
aws:iam/role:Role                        urn:pulumi:staging::kubernetes-worker-nodes::aws:iam/role:Role::role1
aws:iam/instanceProfile:InstanceProfile  urn:pulumi:staging::kubernetes-worker-nodes::aws:iam/instanceProfile:InstanceProfile::instanceProfile1

Found no pending operations associated with staging

Backend        
Name           name
URL            s3://cloud-eks
User           user.name
Organizations  

Dependencies:
NAME        VERSION
pip         23.2.1
pulumi-eks  1.0.3
setuptools  68.1.2
wheel       0.41.2

Pulumi locates its logs in /tmp by default