aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.68k stars 3.92k forks source link

(elasticloadbalancingv2): Cannot add ECS service with multiple containers to ALB target groups #12947

Closed revmischa closed 3 years ago

revmischa commented 3 years ago

I've created an ECS fargate service with three different containers on different ports. I would like to add this ECS service to three different TargetGroups, with the different target groups corresponding to the different services.

Reproduction Steps

You can see the full setup here: https://github.com/jetbridge/lemmy-cdk/tree/ecs-single-service/lib/lemmy

 // target group for lemmy backend
    const backendTg = new ApplicationTargetGroup(this, "LemmyBETargetGroup", {
      vpc,
      protocol: ApplicationProtocol.HTTP,
      port: BACKEND_PORT, // 8536
      targetGroupName: "Lemmy-Backend",
      targetType: TargetType.IP,
      healthCheck: {
        path: "/api/v2/categories",
        interval: Duration.seconds(10),
        healthyThresholdCount: 2,
      },
    });

    // target group for lemmy frontend
    const frontendTg = new ApplicationTargetGroup(this, "LemmyFETargetGroup", {
      vpc,
      protocol: ApplicationProtocol.HTTP,
      port: FRONTEND_PORT,  // 1234
      targetType: TargetType.IP,
      targetGroupName: "Lemmy-Frontend",
      healthCheck: {
        interval: Duration.seconds(10),
        healthyThresholdCount: 2,
      },
    });

    // target group for iframely
    const iframelyTg = new ApplicationTargetGroup(this, "IFramelyTargetGroup", {
      vpc,
      protocol: ApplicationProtocol.HTTP,
      port: IFRAMELY_PORT,  // 8061
      targetType: TargetType.IP,
      targetGroupName: "IFramely-v2",
      healthCheck: {
        interval: Duration.seconds(10),
        healthyThresholdCount: 2,
        healthyHttpCodes: "302",
      },
    });

...

// the containers look like this:
    const container = taskDef.addContainer(IFRAMELY_NAME, {
      image: ContainerImage.fromRegistry(IFRAMELY_IMAGE),
      logging: LogDriver.awsLogs({ streamPrefix: IFRAMELY_NAME }),
    });
    // map port
    container.addPortMappings({
      containerPort: IFRAMELY_PORT,
      protocol: Protocol.TCP,
    });

...

 // containers
    new LemmyFrontend(this, "LemmyFrontend", { taskDef });
    new LemmyBackend(this, "LemmyBackend", { taskDef, db });
    new IFramely(this, "IFramely", { taskDef });
    new Pictrs(this, "Pictrs", { taskDef, assetVolume });

    // service
    const secGroup = new SecurityGroup(this, "SecGroup", { vpc });
    const lemmyService = new FargateService(this, "BackendService", {
      cluster,
      // namespace,
      assignPublicIp: true, // or false, whatever
      taskDefinition: taskDef,
      platformVersion: FargatePlatformVersion.VERSION1_4,
      desiredCount: 1,
      serviceName: `lemmy`,
      // cloudMapOptions: { cloudMapNamespace: namespace, name: "lemmy" },
      securityGroups: [secGroup],
    });
    lemmyService.registerLoadBalancerTargets;

    // all target groups point at our ECS service, on different ports
    const a = lemmyLoadBalancer.frontendTargetGroup.addTarget(lemmyService);
    lemmyLoadBalancer.backendTargetGroup.addTarget(lemmyService);
    lemmyLoadBalancer.iframelyTargetGroup.addTarget(lemmyService);

What did you expect to happen?

What I want is each container to be mapped to each target group. I would expect the ALB to send traffic to the various containers on their respective ports.

What actually happened?

ECS creates three target group associations with only one container on its port.

Screen Shot 2021-02-09 at 8 23 46 PM

Environment

Basically I want to have a single task definition with multiple containers, where the containers each go into their own target groups. Instead the target groups only associate with the first container in the service.


This is :bug: Bug Report

njlynch commented 3 years ago

One thing that jumps out to me is lemmyService.registerLoadBalancerTargets. This is a method, which takes a list of targets to register to the listener; the code is not calling the method, or passing in the targets. You could opt to use that method instead.

That being said, your above approach would work, but you need to use service.loadBalancerTarget. See the docs for details, but this allows you to specify a non-default container as the target:

lemmyLoadBalancer.backendTargetGroup.addTarget(lemmyService.loadBalancerTarget({
  containerName: BACKEND_NAME,
});
github-actions[bot] commented 3 years ago

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

jamesh48 commented 7 months ago

One thing that jumps out to me is lemmyService.registerLoadBalancerTargets. This is a method, which takes a list of targets to register to the listener; the code is not calling the method, or passing in the targets. You could opt to use that method instead.

That being said, your above approach would work, but you need to use service.loadBalancerTarget. See the docs for details, but this allows you to specify a non-default container as the target:

lemmyLoadBalancer.backendTargetGroup.addTarget(lemmyService.loadBalancerTarget({
  containerName: BACKEND_NAME,
});

This ended my headache, I was targeting the service alone and both target groups were going to one port instead of separate ports, heres the change I made, perviously it said targets: [bshFargateService] for both.

const backendTargetGroup = new elbv2.ApplicationTargetGroup(
      this,
      'bsh-be-tg',
      {
        port: 8080,
        protocol: elbv2.ApplicationProtocol.HTTP,
        targets: [
          bshFargateService.loadBalancerTarget({
            containerName: backendContainer.containerName,
          }),
        ],
        vpc: ec2.Vpc.fromLookup(this, 'jh-imported-vpc-tg-be', {
          vpcId: props.aws_env.AWS_VPC_ID,
        }),
        healthCheck: {
          path: '/healthcheck',
          unhealthyThresholdCount: 2,
          healthyHttpCodes: '200',
          healthyThresholdCount: 5,
          interval: cdk.Duration.seconds(30),
          port: '8080',
          timeout: cdk.Duration.seconds(10),
        },
      }
    );

    const frontendTargetGroup = new elbv2.ApplicationTargetGroup(
      this,
      'bsh-fe-tg',
      {
        port: 3000,
        protocol: elbv2.ApplicationProtocol.HTTP,
        targets: [
          bshFargateService.loadBalancerTarget({
            containerName: frontendContainer.containerName,
          }),
        ],
        vpc: ec2.Vpc.fromLookup(this, 'jh-imported-vpc-tg-fe', {
          vpcId: props.aws_env.AWS_VPC_ID,
        }),
        healthCheck: {
          path: '/api/healthcheck',
          unhealthyThresholdCount: 2,
          healthyHttpCodes: '200',
          healthyThresholdCount: 5,
          interval: cdk.Duration.seconds(30),
          port: '3000',
          timeout: cdk.Duration.seconds(10),
        },
      }
    );