aws-samples / foundation-model-benchmarking-tool

Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.
https://aws-samples.github.io/foundation-model-benchmarking-tool/
MIT No Attribution
197 stars 31 forks source link

Allow backup/ failover regions to be specified #232

Open jimburtoft opened 2 weeks ago

jimburtoft commented 2 weeks ago

In the experiments, allow the user to specify region: for example: us-east-1 | us-west-2 Treat this as a config which says try us-east-1 first but then failover to us-west-2 if the deployment fails.

Deployments could fail because of lack of quota or insufficient capacity.

However, if the deployment in the first region succeeds, I do NOT want to deploy into the second region.

(this should also flow through from fmbench orchestrator)