(N/A): Circular dependency between resources when using Fargate profiles.

[javydekoning]

Describe the bug

This Stack was working fine on a previous version of CDK Blueprints (I think 1.5.4 was the latest I deployed).

However, now it's running into a circular dependency error:

Circular dependency between resources: [eksclusterebscsicontrollersasaConditionJson452D9CAA, eksclusterefscsicontrollersamanifestefscsicontrollersaServiceAccountResource611B8BB1, eksclusterAwsAuthmanifest769BDE7D, eksclusterblueprintsaddonkarpentersaConditionJson82E62338, vpccniaddOn, eksclusterchartawsefscsidriverD7A478D7, eksclusterfargateprofilekarpenterD8717A9C, eksclusterebscsicontrollersasamanifestebscsicontrollersasaServiceAccountResourceA0F36911, karpenternamespacestructC6800395, eksclusterefscsicontrollersaRoleA2432411, eksclusterefscsicontrollersaRoleDefaultPolicy55B4D64D, eksclusterblueprintsaddonkarpentersaRoleA1839D1B, eksclusterchartmetricsserverDC1B5C14, externaldnsnsA88F2381, ekscluster92983EFB, eksclusterOpenIdConnectProvider30D7C8A6, corednsaddOn, eksclusterexternaldnsConditionJsonC28E2FA5, eksclusterfargateprofilekarpenterPodExecutionRole1C34BDAD, eksclusterawsloadbalancercontrollerRole5E4944FA, eksclusterblueprintsaddonkarpentersamanifestblueprintsaddonkarpentersaServiceAccountResourceF8F0C2F8, awscdkawseksKubectlProviderNestedStackawscdkawseksKubectlProviderNestedStackResourceA7AEBA6B, eksclusterchartexternaldns63D18F1F, eksclusterebscsicontrollersasaRoleEC26EC03, eksclustermanifestdefaultprovisioner01B7EC7B, eksclusterexternaldnsRole069B1D87, eksclusterawsloadbalancercontrollerRoleDefaultPolicyFE5B446B, kubeproxyaddOn, eksclusterKubectlReadyBarrier2A406583, eksclusterefscsicontrollersaConditionJson374DE8F1, eksclusterchartawsloadbalancercontrollerC40413F5, eksclusterexternaldnsRoleDefaultPolicy3F4CB466, eksclusterawsloadbalancercontrollerConditionJson629E686D, eksclusterawsloadbalancercontrollermanifestawsloadbalancercontrollerServiceAccountResource2BAA4404, eksclusterCreationRoleDefaultPolicy3CACA8C6, eksclusterchartkarpenterAC57B74F, eksclusterexternaldnsmanifestexternaldnsServiceAccountResourceBACF284A, awsebscsidriveraddOn]

If I do not include a Fargate profile the dependency error goes away.

Expected Behavior

Deploy without issues

Current Behavior

Circular dependency error

Reproduction Steps

See this stack:

import { Stack, IResource, StackProps, aws_eks as eks } from "aws-cdk-lib";
import * as blueprints from "@aws-quickstart/eks-blueprints";
import { Construct } from "constructs";
import { IVpc } from "aws-cdk-lib/aws-ec2";
import { IHostedZone } from "aws-cdk-lib/aws-route53";

export interface EksLabStackProps extends StackProps {
  vpc: IVpc;
  zone: IHostedZone;
  certificateArn: string;
}

export class EksLabStack extends Stack {
  constructor(scope: Construct, id: string, props: EksLabStackProps) {
    super(scope, id, props);

    const clusterProvider = new blueprints.GenericClusterProvider({
      version: eks.KubernetesVersion.V1_24 ,
      clusterLogging: [
        eks.ClusterLoggingTypes.API,
        eks.ClusterLoggingTypes.AUDIT,
        eks.ClusterLoggingTypes.AUTHENTICATOR,
        eks.ClusterLoggingTypes.CONTROLLER_MANAGER,
        eks.ClusterLoggingTypes.SCHEDULER,
      ],
      fargateProfiles: {
        karpenter: {
          fargateProfileName: "karpenter",
          selectors: [{ namespace: "karpenter" }],
          vpc: props.vpc
        },
      },
    });

    const addOns: Array<blueprints.ClusterAddOn> = [
      new blueprints.addons.VpcCniAddOn(),
      new blueprints.addons.KarpenterAddOn({
        amiFamily: "AL2",
        ttlSecondsUntilExpired: 60 * 60 * 34 * 7,
        requirements: [
          {
            key: "karpenter.sh/capacity-type",
            op: "In",
            vals: ["spot"],
          },
          {
            key: "kubernetes.io/arch",
            op: "In",
            vals: ["amd64", "arm64"],
          },
          {
            key: "topology.kubernetes.io/zone",
            op: "In",
            vals: ["eu-west-1a", "eu-west-1b"],
          },
        ],
        consolidation: { enabled: true },
        subnetTags: {
          "karpenter.sh/discovery": "eks-lab",
        },
        securityGroupTags: {
          "kubernetes.io/cluster/eks-cluster": "owned",
        },
      }),
      new blueprints.addons.AwsLoadBalancerControllerAddOn(),
      new blueprints.addons.CoreDnsAddOn(),
      new blueprints.addons.EfsCsiDriverAddOn(),
      new blueprints.addons.EbsCsiDriverAddOn(),
      new blueprints.addons.ExternalDnsAddOn({
        hostedZoneResources: [props.zone.zoneName],
      }),
      new blueprints.addons.KubeProxyAddOn(),
      new blueprints.addons.MetricsServerAddOn(),
    ];

    const resourceProviders = new Map<
      string,
      blueprints.ResourceProvider<IResource>
    >([
      [
        blueprints.GlobalResources.Vpc,
        new blueprints.DirectVpcProvider(props.vpc),
      ],
      [
        props.zone.zoneName,
        new blueprints.ImportHostedZoneProvider(props.zone.hostedZoneId),
      ],
      [
        "cert",
        new blueprints.ImportCertificateProvider(
          props.certificateArn,
          "wildcardCert"
        ),
      ],
    ]);

    new blueprints.EksBlueprint(
      this,
      {
        addOns,
        clusterProvider,
        resourceProviders,
        id: "eks-cluster",
      },
      props
    );
  }
}

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.78.0

EKS Blueprints Version

1.7.2

Node.js Version

20

Environment details (OS name and version, etc.)

Mac OS Ventura

Other information

No response

[shapirov103]

I have a fix for and it will be released in 1.7.4. Please don't use an outer stack, it creates a redundant stack and frequently causes issues. See more here.

[javydekoning]

I have a fix for and it will be released in 1.7.4.

Please don't use an outer stack, it creates a redundant stack and frequently causes issues. See more here.

Thanks! If I shouldn't use an outer stack, then how do I best pass an IVpc from another stack in the same App?

[shapirov103]

One example of using DirectVpcProvider is here. Generally, you will have a separate stack for VPC and reuse. You can also look up Vpc with new VpcProvider('my-vpc-id'). If you use dedicated VPC for the cluster, you can register your custom VPC provider with your specific logic right in the blueprint as well as described here.

[javydekoning]

Still have this on 1.8.0 even without using an outer stack.

const clusterProvider = new blueprints.GenericClusterProvider({
  version: eks.KubernetesVersion.V1_26,
  securityGroup: network.sg,
  fargateProfiles: {
    karpenter: {
      fargateProfileName: "karpenter",
      selectors: [{ namespace: "karpenter" }],
    },
  },
});

new blueprints.BlueprintBuilder()
  .addOns(
    ...addOns
  )
  .resourceProvider(
    blueprints.GlobalResources.Vpc,
    new blueprints.DirectVpcProvider(network.vpc)
  )
  .resourceProvider(
    certStack.zone.zoneName,
    new blueprints.ImportHostedZoneProvider(certStack.zone.hostedZoneId),
  )
  .resourceProvider(
    "cert", new blueprints.ImportCertificateProvider(
      certStack.certificateArn,
      "wildcardCert"
    ),
  )
  .clusterProvider(clusterProvider)
  .build(app, "eks-blueprint");

Any workaround available?

[shapirov103]

@javydekoning the issue somehow got reintroduced, apologies about that.

My assumption is that your list of addons contains vpc-cni addon. In that case, here is a workaround that you can use before I release the patch:

.addOns(
                new blueprints.VpcCniAddOn({
                    controlPlaneAddOn: false
                } as blueprints.VpcCniAddOnProps))

Hope it helps, I will take a look at the fix shortly.

[javydekoning]

That worked! Thank you

[shapirov103]

1.8.1 is published now. Appreciate your early feedback. Hope it works this time.

[shapirov103]

Closing, we can always reopen if needed.