[ECS,Fargate] Multiple target groups for a service

[zaitsman]

Summary

I want to be able to route traffic from two different Path rules in my ALB to the same ECS instance and have ECS automatically register targets in both groups.

Description

Pretty much what the summary says. I can manually add targets to the second group, but if the service is restarted, then the second group loses the targets

Expected Behavior

Unable to have 2 targets for the same ECS service

Observed Behavior

N/A

Environment Details

Supporting Log Snippets

[bfutterleib]

+1 our omnidb service needs dual port exposure. Really surprised this wasnt implemented from the start given that multiple port mappings are available out the box for other orchestrators.

[anshul0915zinnia]

+1

[bhkangw]

+1 would love to see progress on this as it's also preventing us from taking advantage of Fargate.

[jk2l]

+1 Hi, in the last 1 year we encountered at least 4 customers that want to use ECS but their containers all require multiple port.

Most common reason is my clients are doing mass migration with lift and shift to move from on-premise VMware into ECS. However this "single port" cause a big issue and the clients are unable to take advantage of it. And unfortunately as AWS advanced partner we would like to promote AWS service instead of adopt 3rd party solution, but to our clients ECS is more like a incomplete solution and it cannot be used at all

[alexandersm]

+1 , our services are listening more than one port and it would be really great to have this feature available in ECS

[vimmis]

+1

[franz-newsadoo]

+1

[herrri]

+1

[coccyx]

+1

[AndreyChugunkovDevPro]

+1

[whereisaaron]

Please show some common courtesy people ✋. No one, including AWS, is going to respect you for posting a '+1' message on Github issue. Anyone but a Github n00b knows you vote-up the original post with a 👍, also known as a '+1' reaction emoji. to show your support. Go ahead and also 👍 any comments above with suggestions or use cases you support, or ❤️ it even! Reaction emoji are available on both the web and mobile clients, so no excuses.

On the other hand, if you have a use case or application that would benefit from this requested feature, or a suggestion on how to implement the change. Do post a comment and share your story.

But please don't waste the time of 50+ people subscribed here by sending everyone a notification with nothing by a '+1' 'me too' post. Instead, go up the top and hit that 👍. I appreciate your support for this issue.

Like and subscribe! 👍 😄

[ashayshub]

@coultn Is this going to be released only for Fargate? That will be a shame really cause I bet most of the users using it are asking it for EC2 as well!

[coultn]

@coultn Is this going to be released only for Fargate? That will be a shame really cause I bet most of the users using it are asking it for EC2 as well!

When this feature is launched it will support ECS on EC2 and ECS on Fargate (commonly known as just 'Fargate').

[chrobotm]

@coultn will this get released with cloudformation support?

[ledbit]

+1

[coultn]

@coultn will this get released with cloudformation support?

Yes, that is the plan.

[akshayram-wolverine]

Hi everyone! I am on the product management team on AWS Container Services. This feature is now in developer preview.

If you are interested in participating please see here: https://github.com/aws/containers-roadmap/tree/master/preview-programs/ecs-multipletg-service In the interest of your time, the link has examples to set up an ECS service with multiple target groups in both the EC2 and Fargate launch type. Please email me at akshram [at] amazon.com with your AWS account ID(s) and AWS region(s) to get started.

We would love to hear feedback!

[whereisaaron]

Thanks for the update @akshayram-wolverine, I read the examples and both the use cases there focus on using multiple ALBs for one ECS/Fargate service. The misses one common use case, i.e. imagine you have an ECS/Fargate service expose ports 80 and 443, and you want to expose both ports with one ALB or ELBv2. Will this new feature be able to support that? Can we have port 80 and 443 on the same Load Balancer going to the same ECS/Fargate service?

[akshayram-wolverine]

Hey @whereisaaron. Yes your use case should be supported as well! You should just create two target groups one for each port and attach it to your ECS service.

[anshul0915zinnia]

Thanks for the update @akshayram-wolverine . can it support the use like if have one container running in service can we link it to 2 target groups which will help us to resolve issue to host our application on internal ELB and external ELB?

On Tue, Jul 9, 2019 at 10:18 AM akshayram-wolverine < notifications@github.com> wrote:

Hey @whereisaaron https://github.com/whereisaaron. Yes your use case should be supported as well! You should just create two target groups one for each port and attach it to your ECS service.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/aws/containers-roadmap/issues/104?email_source=notifications&email_token=AB4BMPJLGHPELHZ5RI57GC3P6QKBLA5CNFSM4GPJVEQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZPCGUY#issuecomment-509485907, or mute the thread https://github.com/notifications/unsubscribe-auth/AB4BMPNIGHZRJU5XM4WPZG3P6QKBLANCNFSM4GPJVEQQ .

-- Regard, Anshul Agarwal Mob No:+91-7830892922

[akshayram-wolverine]

Hey @anshul0915 Yes!

[pitkley]

Hi @akshayram-wolverine. These are great news, thank you for implementing this feature-request!

One question I have is whether or not target-groups of instance-type ip are supported with this change? Right now assigning a network load balancer target-group whose instance-type is set to ip, rather than the default instance, fails with a message noting that only instance-mode is supported.

Our use-case is the ability to run TCP-based services within an ECS-cluster, making these services available through a network load balancer, and have those services be reachable by other services within the same ECS node! The last point is crucial: if the request is originating from the same instance the NLB tries to route to, i.e. the source instance-ID and destination instance-ID are identical, the request will not be able to propagate through the NLB (I can't find the documentation for this right now, unfortunately).

Should this be a concious decision or technical limitation, is there a "workaround" or alternative solution to this problem? (I.e. should one use the awsvpc networking in this case?)

(This question might be hijacking this issue, but I was unable to find a more fitting one. Please let me know if I should open a new issue for this!)

[whereisaaron]

@pitkley: Loopback timeout

[pitkley]

@whereisaaron thank you, that is what I was unable to find. If I understand the points made by the documentation correctly, running the containers with the awsvpc networking-mode should fix my issue. I'll test this as soon as I get around to it and report back.

[pmoghaddam]

Will there be a way to distinguish which ALB the request is coming from?

In the case of internal vs. external, if you cannot tell the difference, it'll be challenging to effectively use this.

[coultn]

Launched! https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-ecs-services-now-support-multiple-load-balancer-target-groups/

[GoeranEnvirio]

Is it possilbe to configure a current fargate service this way: aws ecs update-service --service my-http-service --cli-input-json loadbalancers.json

where I have put the loadbalancers like this is in the loadbalancers.json file:

"loadBalancers":[ {
"targetGroupArn":"arn:aws:elasticloadbalancing:region:123456789012:targetgroup/target_group_name_1/1234567890123456", "containerName":"jenkins", "containerPort":8080 }, {
"targetGroupArn":"arn:aws:elasticloadbalancing:region:123456789012:targetgroup/target_group_name_2/6543210987654321", "containerName":"jenkins", "containerPort":50000 } ]

[GoeranEnvirio]

It did not work got:

Error parsing parameter 'cli-input-json': Invalid JSON: No JSON object could be decoded JSON received: loadbalancers.json

[Istador]

Is it possilbe to configure a current fargate service this way: aws ecs update-service --service my-http-service --cli-input-json loadbalancers.json

where I have put the loadbalancers like this is in the loadbalancers.json file:

[...]

If I remember correctly, the loadBalancers settings can't be changed once the service is created. Neither with the aws ecs update-service command nor with the aws console.

You should be able to create a new service with aws ecs create-service that is using multiple loadBalancers, and then delete the old one.

It did not work got:

Error parsing parameter 'cli-input-json': Invalid JSON: No JSON object could be decoded JSON received: loadbalancers.json

The example in the documentation of create-service is using the following syntax with file://:

--cli-input-json file://ecs-simple-service-elb.json

---

BTW another use case that is working thanks to this feature: Having two target groups for the same ecs service, with one target group being sticky and the other being non-sticky. The public load balancer decides which paths needs to be sticky or not, by forwarding to the designated target group.

[GoeranEnvirio]

@Istador thanks for the info. It's really irritating when it's almost undocumented and you need the feature.

[GoeranEnvirio]

I tried the following: aws ecs create-service \ --cluster TestDevEC2Cluster \ --service-name ecs-multiport-service-nlb \ --launch-type EC2 \ --cli-input-json file://ecs-multiport-service-nlb.json

An error occurred (InvalidParameterException) when calling the CreateService operation: Not support Classic Load Balancer in multiple load balancers.

I have configured an NLB! So what could the issue be?

If I add a role: "AWSServiceRoleForECS"

It will complain: An error occurred (InvalidParameterException) when calling the CreateService operation: You cannot specify an IAM role for services that require a service linked role.

The file:

{ "serviceName": "ecs-multiport-service-nlb", "taskDefinition": "test-ec2task:2", "loadBalancers": [ { "loadBalancerName": "NewTestNLB", "containerName": "test", "containerPort": 80, "targetGroupArn": "xxxxx" }, { "loadBalancerName": "NewTestNLB", "containerName": "test", "containerPort": 2022, "targetGroupArn": "xxxxx" } ], "desiredCount": 1 }

So how can I create it without an error?

SUCCEDED!

There was a lot of rabbit holes and hidden ramifications. Probably the reason the examples do not work. The only way to fix it is to read a lot.

[quolpr]

@GoeranEnvirio the issue is that you specified loadBalancerName in loadBalancers. Only targetGroupArn, containerName, containerPort should be specified

The only way to fix it is to read a lot.

But the better way - describe how did you fix your issue, so that will save a lot of time of other guys ;)

[stnor]

I have a use case where I would need a service to use two target groups. One is an NLB which forwards requests (SSL client cert) to one port on the service, and another is used on an ALB.

I'm unable to configure this in the web console. and according to support (Case ID - 6524068251), this is not supported. Please consider this use case in the future, thanks. As it is right now, I need to run two services that does the same thing.

Thanks!

[wedneyyuri]

@stnor it's really strange because your usecase is already supported but you will need to recreate your ECS service.

[stnor]

@wedneyyuri According to AWS Support, a mix of NLB and ALB target groups isn't supported.

[kstro21]

According to the documentation here

The service must use either an Application Load Balancer or Network Load Balancer.

We can't use both for the same service. I would benefit from being able to use both. Please, consider this use case.