search

insitro / redun

Yet another redundant workflow engine
https://insitro.github.io/redun/
Apache License 2.0
510 stars 43 forks source link

Specify vcpu and memory as resourceRequirements in AWS Batch executor #95

Closed aotimme closed 3 months ago

aotimme commented 3 months ago

When trying to use redun with AWS Batch on Fargate using the existing AWSBatchExecutor, we get the following error:

botocore.errorfactory.ClientException: An error occurred (ClientException) when calling the SubmitJob operation: vcpus is not applicable for Fargate. Use resource requirements instead.

As it turns out, it looks like specifying vcpu and memory directly on containerProperties is now deprecated (see here for vcpu and here for memory) in favor of specifying them in the resourceRequirements array, so that's the update provided here.

This update will allow running on Fargate with the AWSBatchExecutor.

mattrasmus commented 3 months ago

Hi @aotimme thanks for sharing this PR! We'll test it out and merge it in.