[ECS] [request]: Support Cross Service Cross Task Definition Dependency

[e-moshaya]

Tell us about your request What do you want us to build? Support Cross Service Cross Task Definition Dependency

Which service(s) is this request for? Fargate, ECS

Based on aws documentation https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html,

"Your entire application stack does not need to exist on a single task definition, and in most cases it should not. Your application can span multiple task definitions by combining related containers into their own task definitions, each representing a single component"

This means that it is best practice to separate each component/service into its own task definition. However, cross task definition is not yet supported. Only container dependencies within a single task definition is supported. This needs extending to support cross service.

[coultn]

Can you explain more about your use case? Do you want to have services that start up and shut down in a certain order? Or something else? The more detail you can provide, the better we can evaluate and prioritize.
Thanks for your feedback!

[e-power]

+1 for this.

As you said @coultn we would like to prioritize startup order of different services which can run into issues if a certain service starts before the other (currently random). This has come up basically as a result of starting and stopping the ECS cluster overnight for cost saving reasons.

[vsnig]

Need this to run Apollo GraphQL Federated Schema. In short, there is a Gateway that polls child schemas (Fargate services) on startup to compile a single schema. The Gateway must spawn after all child services are up. Otherwise it won't contain parts of schema that were spawned after it.

[kamatama41]

We have daemon services of datadog-agent on ECS EC2 clusters. As the agent task collects metrics and logs from other tasks on an EC2 instance, it should start before (and stop after) any other tasks when an EC2 instance is initializing (and draining). It would be achieved if we can define the dependency between services.

[mooreniemi]

K8s has initContainers pattern. Looks like this can be done with Container Depdencies in ECS.

[afdecastro879]

HI all! Is there any findings around this ticket? Our use case is we want to run a setup task before we spin up our tasks in ECS, the problem is, this setup task should be run only once even if we want to spin up more than one task for our application. I've checked the Container Dependencies in ECS and this doesn't work as it will run the initial container for each instance of our app. As far as I understand the purpose of the ticket is how to make sure independent tasks can depend on each other at the service level and not at the container level

[sjohnsonsf]

+1 vote for this feature. Use case is a Rails app, running a task to completes db:migrate first, then running the app server after migrations have completed.

[jas550]

+1 vote for this feature. Use case is a php app, first the db migration task needs to run once and then php app should execute after migrations have completed successfully.

[eeepmb]

+1 for the same reason as @afdecastro879

HI all! Is there any findings around this ticket? Our use case is we want to run a setup task before we spin up our tasks in ECS, the problem is, this setup task should be run only once even if we want to spin up more than one task for our application. I've checked the Container Dependencies in ECS and this doesn't work as it will run the initial container for each instance of our app. As far as I understand the purpose of the ticket is how to make sure independent tasks can depend on each other at the service level and not at the container level

[pbudzon]

This is also important when you add in the fact that ECS has a limit on task definition size (65536 bytes). This means that even if you tried to put in your application within one task (to use the DependsOn from task definition), it quickly becomes impossible because you hit the size limit.

[TreehouseFalcon]

+1, this issue has been open for about 5 years now and I'm shocked it isn't natively supported in ECS. For my use case, I have a postgres and redis database that need to be deployed first and then my dependent services afterwards. Currently when cold-starting all services in a cluster via EC2, the API and consumer tasks hog available space on my larger instance. This means that I can't utilize the full space on that instance for redis/postgres and need to micromanage task startups until the database tasks start on the right instance.

[steven-geo]

+1 Here. I have a service container that other containers need to connect to, but are not directly part of that core service. Using DependsOn seems to only work internally in the same service/task, not to other services. Adding an option to Depend on a service would be great, with the same condition options as DependsOn.