[ECS] [request]: Update (add/remove) ALB TargetGroups on a ECS Service that has been already created

[gfysaris]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request After an ECS Service is created there is no option to update/modify the ALB TargetGroups that are integrated with the service. In case there is a need to change the alb, running ECS Services need also to be replaced.

Which service(s) is this request for? ECS (Both over ASG or Fargate)

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? We would like to have the ability to change the ALB and the TargetGroups that are integrated with an ECS Service that is already created previously.

Are you currently working around this issue? Currently, when we introduce a new ALB (and new TargetGroups for that ALB) we provision new ECS Services from scratch.

Additional context Anything else we should know? - No, nothing else.

Attachments -NONE-

[zbintliff]

One use case here is for custom deployment controllers. By allowing a service update we can attach it to a non production load bearing TG, run smoke tests, then move it over to the production TG attached to the ALB/NLB

[zivyatziv]

More information I can add - While updating a service's network configuration and adding a TG, you get an error: Load balancing settings can only be set on service creation.

Documentation is also misleading:

• https://docs.aws.amazon.com/cli/latest/reference/ecs/update-service.html
• https://docs.aws.amazon.com/AmazonECS/latest/developerguide/register-multiple-targetgroups.html

But it is described in ECS "Updating a service": https://docs.aws.amazon.com/AmazonECS/latest/developerguide/update-service.html ^ Under the important section, I quote:

If your service uses a load balancer, the load balancer configuration defined for your service when it was created cannot be changed. If you update the task definition for the service, the container name and container port that were specified when the service was created must remain in the task definition.

To change the load balancer name, the container name, or the container port associated with a service load balancer configuration, you must create a new service.

Amazon ECS does not automatically update the security groups associated with Elastic Load Balancing load balancers or Amazon ECS container instances.

Question is about the 3rd point I've quoted (4th in the page), why? :(

[andrewloux]

This is one of the biggest painpoints using ECS on Fargate

[anujkumar-df]

Can anyone try detaching the target group from one load balancer and attaching it to another load balancer? I just found that using this method, you can basically change the load balancer endpoint. You don't even need to modify the existing ECS service.

[AbhishekNautiyal]

Amazon ECS UpdateService API now supports updates for the following parameters: loadBalancers, propagateTags, enableECSManagedTags, and serviceRegistries; see what's new post for details:

https://aws.amazon.com/about-aws/whats-new/2022/03/amazon-ecs-service-api-updating-elastic-load-balancers-service-registries-tag-propagation-ecs-managed-tags/

[kevinkupski]

When updating the Load Balancer field and using a classic load balancer, I get InvalidParameterException: Target Group Arn can not be blank.. Is this something you see as well?

[AbhishekNautiyal]

When updating the Load Balancer field and using a classic load balancer, I get InvalidParameterException: Target Group Arn can not be blank.. Is this something you see as well?

Hi @kevinscholz could you share more details about your configuration. Are you trying to add a load balancer to a service that did not have one previously? It would be great if you could also DM me your service details and updateService payload. Thanks.

[kevinkupski]

It appeared when updating the Terraform AWS Provider. One of the tests failed, which basically first creates a classic load balancer and attaches it to a service and then updates the container name and container port in the load balancer attachment.

The interesting part is this:

resource "aws_ecs_service" "test" {
  name            = %[1]q # This is fix and does not change
  cluster         = aws_ecs_cluster.test.id
  task_definition = aws_ecs_task_definition.test.arn
  desired_count   = 1
  iam_role        = aws_iam_role.ecs_service.name
  load_balancer {
    elb_name       = aws_elb.test.id
    container_name = %[3]q # this changes during the update from "ghost" to "nginx"
    container_port = %[4]d # this changes during the update from "2368" to "80"
  }
  ...
}

Do you need additional information? Not sure how to reach you via DM.

[AbhishekNautiyal]

It appeared when updating the Terraform AWS Provider. One of the tests failed, which basically first creates a classic load balancer and attaches it to a service and then updates the container name and container port in the load balancer attachment.

The interesting part is this:

resource "aws_ecs_service" "test" {
  name            = %[1]q # This is fix and does not change
  cluster         = aws_ecs_cluster.test.id
  task_definition = aws_ecs_task_definition.test.arn
  desired_count   = 1
  iam_role        = aws_iam_role.ecs_service.name
  load_balancer {
    elb_name       = aws_elb.test.id
    container_name = %[3]q # this changes during the update from "ghost" to "nginx"
    container_port = %[4]d # this changes during the update from "2368" to "80"
  }
  ...
}

Do you need additional information? Not sure how to reach you via DM.

Hi @kevinscholz We're looking into it. We'd need some more details to investigate (viz. account number, region, timeframe of failure). Could you please send this info over an email at nautiya[at]amazon[.]com or open a ticket with AWS Support and share the ticket ID with me?

[genbit]

The issue is happening when updating a service using CloudFormation (CFN). The reference to TargetGroup is not removed. I'm reopening the issue to track the related CFN improvements in one place.

[bvtujo]

For customers using the CDK to manage ECS services, you can override the default behavior which sets an empty LoadBalancers field to undefined.

From a cloudformation template:

declare const yourServiceResourceName string;
const template = new cdk.cloudformation_include.CfnInclude(scope, 'Template', {
  templateFile: 'path/to/template.yml',
});
const svc = this.template.getResource(yourServiceResourceName) as ecs.CfnService;
svc.loadBalancers = [];

Or from an existing CDK construct:

declare const stack: cdk.IStack;
declare const cluster: ecs.ICluster;
declare const taskDefinition: ecs.IFargateTaskDefinition;
const svc = new ecs.FargateService(stack, 'FargateService', {
  cluster,
  taskDefinition,
});

(svc.node.defaultChild as ecs.CfnService).loadBalancers = [];

[monelgordillo]

This becomes an issue with ECS, load balancers, and Code Deployment Groups.