awslabs / fargatecli

CLI for AWS Fargate
Apache License 2.0
893 stars 114 forks source link

Parameterized or Environment Variable Fargate Task Launches? #31

Closed MedLexAI closed 6 years ago

MedLexAI commented 6 years ago

Yet another question :)

I have an ECR container that handles document processing, and the goal is to have an S3 upload of a document trigger a Lambda function, which in turn launches a parameterized Fargate task using the name of the document uploaded to S3.

I have an Amazon Linux development AMI setup where I could compile and include the fargate CLI within the Lambda bundle. So the million dollar question is, could your fargate CLI be used to launch a parameterized Fargate task using the name of the file uploaded to S3, perhaps using environment variables such as those in the advanced configuration options for Fargate task definitions?

jpignata commented 6 years ago

Hmm, you should be able to do something like this:

fargate task run <task name> --image <image uri> --env S3_PATH=s3://bucket/key

Was there an issue with that beyond the need to pass --env?

MedLexAI commented 6 years ago

Just the S3 path like you describe would work perfectly as long as it gets set within the Fargate task. I will test this and share my results.

MedLexAI commented 6 years ago

Ok so another issue. Apparently Docker images don't support the standard init runlevel process, so executing the various bits of functionality needed to process the passed S3_PATH environment string is not possible.

I looked a the documentation for fargate task run but didn't see any option to specify a command with the Fargate task launch. Without support for the init process. how do you launch programs within the container using fargate task run? I am trying to avoid ENTRYPOINT or CMD as the image I am using has been modified heavily from a Docker hub pull and not built from a Dockerfile.

jpignata commented 6 years ago

I'm going to be adding a --command flag in the next release as per this feature request: https://github.com/jpignata/fargate/issues/7.

MedLexAI commented 6 years ago

It looks like ECS supports passing COMMAND or ENTRYPOINT:

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definition_environment

But it looks like maybe fargate cli has implemented just the --env option as an equivalent docker run --env?

Could CMD or ENTRYPOINT JSON be passed with --env, or is that implemented strictly for environment variables only per docker run --env?

jpignata commented 6 years ago

Yeah, --env is strictly for environment variables. --command is planned for the next release as I mentioned above.

MedLexAI commented 6 years ago

Cool, thanks for the info. Back to building from a Dockerfile it is then.

kwent commented 6 years ago

Needs --command so badly as well

jpignata commented 6 years ago

Thanks @kwent! I will be adding this in the next release. I have a fair amount of refactoring to complete to get 0.3.0 done, and I will get it out as soon as I can.