search

PrefectHQ / prefect-aws

Prefect integrations with AWS.
https://PrefectHQ.github.io/prefect-aws/
Apache License 2.0
84 stars 40 forks source link

Accept `None` as an argument to Launch Type for ECSTask with `publish_as_work_pool` #362

Closed mattalhonte-srm closed 5 months ago

mattalhonte-srm commented 9 months ago

ECS doesn't let you use both a Capacity Provider and a Launch Type. We use Capacity Providers in order to specify specific Instance Types (r6i.large etc) for our jobs. This works fine if you pass None to the launch_type argument, but when I tried to use one as a foundation for publish_as_work_pool, it gave me this error "Validation failed for field 'launch_type'. Failure reason: None is not one of ['FARGATE', 'EC2', 'EXTERNAL', 'FARGATE_SPOT']". This also works with ECSTask normally at the moment.

Relevant Issue from Prefect 1 that lead to support for Capacity Providers in ECS: https://github.com/PrefectHQ/prefect/issues/5210

Expectation / Proposal

Change the validation so that launch_type=None is okay when using publish_as_work_pool.

Traceback / Example

ecs_task_block = ECSTask(**full_ecs_args)

ecs_task_block.publish_as_work_pool("ecs_task_block_test")

"Validation failed for field 'launch_type'. Failure reason: None is not one of ['FARGATE', 'EC2', 'EXTERNAL', 'FARGATE_SPOT']"
mattalhonte-srm commented 6 months ago

This would fix it too I think (cuz ultimately it's about being able to use Capacity Providers) https://github.com/PrefectHQ/prefect-aws/pull/312