There is no way to pass secrets to ECSTask

[austinweisgrau]

It is possible to pass environment variables as a container override in various ways. For example, if an ECSTask is saved as a block and used as deployment infrastructure, environment variables can be defined as part of the block configuration on the "environment" attribute or using the task customizations.

However, it seems impossible to pass secrets to an prefect_aws.ECSTask.

Normally, an ECS task definition can be defined with a secrets block which can take either plaintext secret values, or be defined with a reference to a secret stored in AWS SSM or AWS Secrets Manager (see here).

If a task definition is defined in ECS with a secrets block, and that task definition arn is passed to a prefect_aws.ECSTask instance, those secrets are not passed to the container that runs the prefect flow. Instead, a new container that was not defined in the ECS task definition is passed as an override to the ECS task, and secret values are not included. Indeed, it is not possible to pass secrets to a container using a container override (see here).

I attempted to hack a solution by passing an override to the ECSTask block to make the override container name match the already-defined container name, so that the prefect flow would be run in the existing container defined with a secret already configured. However, the ECSTask could not start (Task failed to start, exit code 0, nothing in the logs).

Expectation / Proposal

I'm not sure what a solution here is. A few ideas:

• When passed a task definition or task definition arn, pass the command to an existing container already defined on that ECS Task. The environment and secrets already defined for that task definition will be available in the container running the command.
  • If registering a new task definition, enable passing secrets instead of using container overrides to pass plaintext environment variables. I don't think this will work, because for some reason boto3 can't pass a secrets block when registering a task definition (see here) even though that's possible using other ways of interaction with AWS.

Traceback / Example

• [ ] I would like to help contribute a pull request to resolve this!

[austinweisgrau]

Relatedly, it's not ideal that the Prefect API key is passed to the ECS task as a plaintext environment value.