awslabs / fmbench-orchestrator

Run FMBench simultaneously across multiple Amazon EC2 machines to benchmark an FM across multiple serving stacks simultaneously
https://aws-samples.github.io/foundation-model-benchmarking-tool/
2 stars 2 forks source link

Dynamically figure out the AMI + region where instances are deployed #20

Closed madhurprash closed 2 weeks ago

madhurprash commented 3 weeks ago

This PR contains the following changes:

  1. AMI mapping: Users do not have to specify the ami-id anymore. If the "region" parameter is not provided in the instance dictionary, the instance is deployed in the global region where the orchestrator is hosted. If the region is specified, then the ami associated to that region and instance type (gpu/neuron) is spun up.

  2. Configuration file updates & other code refactoring

[Did not use Jinja for this because it was making the code more complex to render and replace the region and then loop through the instances to replace the ami id]

This is tested on llama3.1 8b on a g6e in different regions + llama3.1 70 on inf2/trn1 in us-west-2 and us-east-2.