compose-x / ecs_composex

Manage, Configure and Deploy your services and AWS services and applications from your docker-compose definitions
https://docs.compose-x.io
Mozilla Public License 2.0
158 stars 16 forks source link

add support for ECS Service Connect #727

Closed slootmaekersdirk closed 3 months ago

slootmaekersdirk commented 6 months ago

Is your feature request related to a problem? Please describe. At the moment inter service communication is based on DNS lookup which is not ideal

Describe the solution you'd like Amazon ECS Service Connect allows to communicate between ECS services in an easy way, not using DNS lookup

Additional context What we need to add is

  1. when creating the cluster, specify the --service-connect-defaults parameter
  2. while creating the service add the following properties:
JohnPreston commented 6 months ago

Hello :) Thanks for opening this FR. In the meantime I'd recommend to make sure to use x-cloudmap to create DNS entries etc. for your services. ECS Connect is an "easy build" for what you'd get with Cloud Map and AppMesh iirc Will look into it.

slootmaekersdirk commented 5 months ago

Hello John,

did you find time for this FR? We're about to deply our solution, and this part is still a manual configuration job, after the deploy.

thx Dirk

JohnPreston commented 5 months ago

Hello I did, given that cloudmap etc. is already in compose-x it shouldn't be too much work to implement, I will need to test it out etc. though. That's probably something that will come under the x-network settings to logically map services.

slootmaekersdirk commented 4 months ago

Hello John,

is this FR already in the 0.25 rc?

and , if so, how can I use it?

thx Dirk

JohnPreston commented 4 months ago

Hi @slootmaekersdirk That is not in the 0.25, that will be a 1.x thing The Connect Config is actually quite involved, although I already have all the components in place to make the integration easy, I need to play with finding a good UX for users to configure it.

If you have any suggestion of what you feel the format should look like please do tell.

Otherwise, I might simply allow to define the entire property CFN style in x-ecs_connect or x-network.ecs_connect, but that means more work for the user.

PS: You can already achieve the http://test by using x-cloudmap today without any of the connect faff, if you set the VPC search domain to include the cloudmap domain.

JohnPreston commented 4 months ago

Hello John,

is this FR already in the 0.25 rc?

and , if so, how can I use it?

thx Dirk

What are the properties that you today configure via the UI or in some other way? What does the connect config look like when you do ecs describe-services ?

slootmaekersdirk commented 4 months ago

Hello John,

For me the format of the syntax of ecs describe-services Is enough for me, we don’t need a UX at the moment

Dirk

From: John Preston @.> Date: Tuesday, 5 March 2024 at 07:28 To: compose-x/ecs_composex @.> Cc: Dirk Slootmaekers @.>, Mention @.> Subject: Re: [compose-x/ecs_composex] add support for ECS Service Connect (Issue #727)

Hi @slootmaekersdirkhttps://github.com/slootmaekersdirk That is not in the 0.25, that will be a 1.x thing The Connect Confighttps://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ecs-service-serviceconnectconfiguration.html is actually quite involved, although I already have all the components in place to make the integration easy, I need to play with finding a good UX for users to configure it.

If you have any suggestion of what you feel the format should look like please do tell.

Otherwise, I might simply allow to define the entire property CFN style in x-ecs_connect or x-network.ecs_connect, but that means more work for the user.

— Reply to this email directly, view it on GitHubhttps://github.com/compose-x/ecs_composex/issues/727#issuecomment-1978049953, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACNLQBTD4HC7XR5DAIIFRFLYWVQYTAVCNFSM6AAAAABAPJJFGSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZYGA2DSOJVGM. You are receiving this because you were mentioned.Message ID: @.***>

JohnPreston commented 4 months ago

I am adding the necessary to make it work in the 1.0 branch as this requires things that weren't originally in the docker compose specification but now are EDIT: Likely to be an early access feature in 1.0

slootmaekersdirk commented 3 months ago

Hello John,

please keep me up te date, when it is available for testing

thx Dirk

JohnPreston commented 3 months ago

Planned for 1.1.0 :)

PS: Next week I am at a summit, but should be done and delivered by end of March :) EDIT: insert T&C that say I have no legal obligations to, limited to availability :D

JohnPreston commented 3 months ago

@slootmaekersdirk I have it done in a new feature branch, however, want to check with you that I am doing the right thing. I have the service connect configured, it created an entry in cloudmap but not DNS, API calls only, which is odd. Then the other thing is, I don't see how you declare that service A wants to connect to service B :thinking: I am missing something?

See #745

slootmaekersdirk commented 3 months ago

Hello John,   The whole idea of the connect service is not to go via a DNS, but acting as a side car for each service, so a direct communication between services is possible.   To allow 2 side communication you should (when creating the service):   "serviceConnectConfiguration": {         "enabled": true,         "namespace": "svc-namespace",         "services": [             {                 "portName": "testtask",                 "clientAliases": [                     {                         "port": 80                     }                 ],             }         ],         "logConfiguration": {             "logDriver": "awslogs",             "options": {                 "awslogs-group": "/ecs/service-connect-proxy",                 "awslogs-region": "region",                 "awslogs-stream-prefix": "service-connect-proxy"             }         }     },   Here we create a namespace called svc-namespace I can access this taks via http://testtask:80 all services in the same namespace can communicate with eachother Dirk

JohnPreston commented 3 months ago

Hey @slootmaekersdirk This is now implemented in #745 - if you would please test for me. I tested with the YELB demo app that has been re-used in multiple places to demo connect, and I could get the UI online, vote, and get the stats going up/down Let me know how you get on.

JohnPreston commented 3 months ago

Implemented in #745

JohnPreston commented 3 months ago

See https://pypi.org/project/ecs_composex/1.1.0rc1/