aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.7k stars 3.93k forks source link

[aws-eks] Add EKS Cluster Lookup method #7825

Open juanjoDiaz opened 4 years ago

juanjoDiaz commented 4 years ago

Enable EKS cluster lookups simular to VPC lookup (using only the ID or the ARN). Currently, the closes thing is fromClusterAttributes which requires information about the VPC, the security groups, etc, which seems really unnecessary to me

https://github.com/aws/aws-cdk/blob/be6666b0fe5743c9f8a7747768101fb86380eb38/packages/%40aws-cdk/aws-eks/lib/cluster.ts#L59-L90

Use Case

I am creating an EKS cluster using CDK. In a separate stack, I would like to add a Fargate profile. I have no easy way to get the cluster so I can pass it to the FargateProfile constructor. There is no easy way to do so.

Proposed Solution

A similar solution as what's used in the VPC module

https://github.com/aws/aws-cdk/blob/c4757831bc5ccb8ac510694d083f17d39a423033/packages/%40aws-cdk/aws-ec2/lib/vpc.ts#L993-L1032


This is a :rocket: Feature Request

samuelthan commented 4 years ago

love to see this feature. Helps me with trying to get the default security group that was created by AWS EKS.

At the moment i'm using boto3 to pull this information which seems a bit ugly in the cdk world.

Vote +1

juanjoDiaz commented 4 years ago

Any progress on this?

As said, I'd be happy contributing this feature but I'll probably need some guidance on how to implement it so it's consistent with the other lookups in the library.

eladb commented 4 years ago

Related to #8608

rcollette commented 4 years ago

I am having a similar issue trying add a Fargate service in a CDK application that is separate from the CDK application that creates the cluster. I'm using a combination of ssm parameters using a naming convention to store cluster properties, and Cluster.fromClusterAttributes(), but now I'm running into.

Cannot enable service discovery if a CloudMap Namespace has not been created in the cluster.

The defaultCloudMapNamespace is an object that cannot be serialized to JSON due to cyclical references and it cannot be reconstructed (or at least it seems difficult to do so) because it is not a pure interface, having properties for Stack and Node classes.

If a Cluster instance can be obtained with Cluster.fromLookup() and it's properties including the defaultCloudMapNamespace populated, it would enable creating a Fargate service with CloudMap enabled.

sunshineo commented 3 years ago

Our use case need to allow EKS cluster to access an RDS. We tried to have the RDS creation and allow access in a separate repository, but the EKS cluster we get from fromClusterAttributes did not work. And it failed silently

const eksCluster = eks.Cluster.fromClusterAttributes(this, 'cdk-managed-eks-cluster', { clusterName: 'cdk-managed-eks-cluster', })
const rdsCluster = new rds.DatabaseCluster(this, 'Database', {...});
rdsCluster.connections.allowFrom(eksCluster.connections, rdsPort, 'Allow eks cluster to connect to rds')

The code above works if we create the eks cluster in the same stack.

iliapolo commented 1 year ago

This issue has been re-classified as p2. That means a workaround is available or it is deemed a nice-to-have feature. Given the amount of work there is to do and the relative priority of this issue, the CDK team is unlikely to address it. That does not mean the issue will never be fixed! If someone from the community submits a PR to fix this issue, and the PR is small and straightforward enough, and meets the quality bars to be reviewed and merged with little effort we will accept that PR. PRs that do not build or need complex or multiple rounds of reviews are unlikely to be merged and will be closed to keep our backlog manageable.

We use +1s on this issue to help prioritize our work, and are happy to re-evaluate the prioritization of this issue based on community feedback. You can reach out to the cdk.dev community on Slack to solicit support for reprioritization.

petteriv-nordic commented 1 year ago

@iliapolo What is the workaround you mentioned above?

iliapolo commented 1 year ago

@petteriv-nordic

That means a workaround is available or it is deemed a nice-to-have feature.

This currently is classified as a nice-to-have, no workaround available.

taabrcr commented 1 year ago

Vote +1 from my side as well, as this is a very much needed feature

mauritz-lovgren commented 1 year ago

When creating an ECS service in a separate CDK project from the project that deploys the ECS Cluster (typically if the latter sets up 'general aspects' for the environment and the former creates service specific resources), the defaultCloudMapNamespace created in the environment CDK project is only available to the service CDK project by knowing a lot of details of the namespace defined in the latter.

One might expect that the defaultCloudMapNamespace was available after look up of Cluster in the 'service CDK project' when using Cluster.fromClusterAttributes or Cluster.fromClusterArn?

Currently, we need to do the following when looking up the cluster to provide the namespace to an ECS service:

const cluster = ecs.Cluster.fromClusterAttributes(this, 'Cluster', {
  clusterName: clusterName,
  vpc: vpc,
  defaultCloudMapNamespace: cloudmap.PrivateDnsNamespace.fromPrivateDnsNamespaceAttributes(
      this, 'Namespace', {
        namespaceArn: Arn.format({
          service: 'servicediscovery',
          resource: 'namespace',
          resourceName: namespaceId
        }, this),
        namespaceId,
        namespaceName
      }
});
nikolic-milan commented 8 months ago

Hello,

has there been any progress on this? We have a use case where we have a repository that defines and sets up the cluster infrastructure, and then other repositories for different apps we want to deploy in the cluster. Currently, I can't figure out a way to add the app to the cluster from a different repository. Using the from_cluster_attributes to get the cluster, I can't perform the add_manifest operation. I tried defining the kubectl_role_arn property that is required to enable kubectl execution, but there is always an issue with the roles I add, and I tried all the roles that are connected to the cluster, which have an arn.

Am I missing something, do I need to create a specific role for this? Or is it just not currently possible?

What about the add_helm_chart method, will that work on a cluster found from arn?

Kind Regards, Milan Nikolić

HumbleBeck commented 2 months ago

Hi, this might have been overlooked, but are there any plans to implement this?

ThePlenkov commented 1 month ago

But isn't it what you've been looking for? https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_eks.Cluster.html#static-fromwbrclusterwbrattributesscope-id-attrs

static fromClusterAttributes(scope, id, attrs)
public static fromClusterAttributes(scope: Construct, id: string, attrs: ClusterAttributes): ICluster
Parameters

scope [Construct](https://docs.aws.amazon.com/cdk/api/v2/docs/constructs.Construct.html) — the construct scope, in most cases 'this'.
id string — the id or name to import as.
attrs [ClusterAttributes](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_eks.ClusterAttributes.html) — the cluster properties to use for importing information.
Returns

[ICluster](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_eks.ICluster.html)
Import an existing cluster.
github-actions[bot] commented 20 hours ago

This issue has received a significant amount of attention so we are automatically upgrading its priority. A member of the community will see the re-prioritization and provide an update on the issue.

ThePlenkov commented 14 hours ago

But isn't it what you've been looking for? https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_eks.Cluster.html#static-fromwbrclusterwbrattributesscope-id-attrs

static fromClusterAttributes(scope, id, attrs)
public static fromClusterAttributes(scope: Construct, id: string, attrs: ClusterAttributes): ICluster
Parameters

scope [Construct](https://docs.aws.amazon.com/cdk/api/v2/docs/constructs.Construct.html) — the construct scope, in most cases 'this'.
id string — the id or name to import as.
attrs [ClusterAttributes](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_eks.ClusterAttributes.html) — the cluster properties to use for importing information.
Returns

[ICluster](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_eks.ICluster.html)
Import an existing cluster.

i honestly don't understand why it was downvote by peers. I see this method is importing an existing cluster. Also, ClusterAttributes need only name , rest is optional. https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_eks.ClusterAttributes.html Doesn't it work?