search

awslabs / fargatecli

CLI for AWS Fargate
Apache License 2.0
893 stars 114 forks source link

Fargate Task Definition Increment Issue #33

Open MedLexAI opened 6 years ago

MedLexAI commented 6 years ago

Ok here's another question. Each time I run a task:

$ fargate task run testapp --verbose --image 1234567890.dkr.ecr.us-east-1.amazonaws.com/testapp

...the task definition for testapp is incrementing with each run (e.g. testapp:1, testapp:2 etc).

Is this standard behavior to create a new task definition with each call? Is there any way to just use testapp:1 for each invocation of fargate task run?

jpignata commented 6 years ago

Each invocation of task run creates a new task definition as task definitions are immutable. The CLI doesn't check previous versions of task definitions to see if one matches. I could conceivably allow the option to pass a specific task definition option, but the goal of the project is to try to hide this from the user and I'm not sure how much value it provides. Happy to hear your thoughts to the contrary though! Thanks.

MedLexAI commented 6 years ago

Well our application is using Lambda functions to trigger Fargate tasks which will be called several times per day (sometimes the task would run longer than the 300 second limit of Lambda else I would just do everything in Lambda).

So conceivably this is going to result in hundreds of active task definitions being created every few days due to a new task definition being created with each fargate task run.

I think with the per-second billing model of Fargate, that using it for longer duration tasks such as those that can't quite be accomplished with just Lambda would be a commonplace use case.

jpignata commented 6 years ago

Got it. That will be problematic as ECS will rate limit you to one task definition a second... Hmmm. For now you may just want to use the SDK to call RunTask directly with your task definition. Perhaps we can support --task-definition with overrides...

MedLexAI commented 6 years ago

Ruh roh. I was not aware of this ECS task definition rate limit. So does this rate limit apply only to creating new task definitions, or with RunTask as well for already existing task definitions?

jpignata commented 6 years ago

Just new task definitions. See https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service_limits.html.