Adding support for spot instances

outerbounds / terraform-aws-metaflow

Deploy production-grade Metaflow cloud infrastructure on AWS

https://registry.terraform.io/modules/outerbounds/metaflow/aws/latest

Apache License 2.0

58 stars 50 forks source link

Adding support for spot instances #54

Open wrbooth opened 1 year ago

wrbooth commented 1 year ago

Changed the following to allow for spot instance support:

added ec2_spot and fargate_spot to batch_type variable
added a splotfleet iam role (generated when a spot batch_type is selected)
refactored compute environment type selection

Tested on my own environment via ec2_spot. I did not test fargate_spot.

Testing Screenshots

Functioning compute environment

Successful Metaflow run on spot ec2 instances

dkarlo2 commented 1 year ago

Hi there, what's the status of this PR? I'm also looking for spot instances support (fargate_spot in my case). Is there anything I can do to speed this up?