search

openclimatefix / metoffice_ec2

Subset Met Office MOGREPS-UK and UKV on AWS EC2
MIT License
11 stars 2 forks source link

Look into running workload on Spot instead of Fargate #36

Closed flowirtz closed 4 years ago

flowirtz commented 4 years ago

We are currently running 2 replicas on Fargate for this script. Our compute workload will likely increase. We should look into maybe switching this to spot instances via spot.io Elastigroup.

update. seems like we are a bit ahead of our time. Migrating to spot is very hard. Their docs are significantly outdated. We should not use them for now, imo.

There is however Fargate Spot. As our workloads are fault-tolerant we should definitely look into that.

Seems fairly straightforward, just need to create a capacity provider and make the tasks/service use it.

flowirtz commented 4 years ago

See db5b7e6b5649e80411b04c7e764a437da35f9b1e

flowirtz commented 4 years ago

That was shockingly easy: We are now running the whole workload on spot instances as of db5b7e6. 🎉 🚀