awslabs / fmbench-orchestrator

Run FMBench simultaneously across multiple Amazon EC2 machines to benchmark an FM across multiple serving stacks simultaneously
https://aws-samples.github.io/foundation-model-benchmarking-tool/
2 stars 2 forks source link

code addition to handle for pre existing instances #17

Closed madhurprash closed 3 weeks ago

madhurprash commented 3 weeks ago

This PR contains the following:

  1. If a user wants to benchmark on a pre existing instance that does not have FMBench installed, they would have to stop the instance, update the user data and re run the instance. Now in this case, the start up script is directly run on the pre existing instance before running the benchmarking test.

  2. Made the key pair name independent of the region in the case of when a user wants to benchmark on a pre existing instance. If a user has already configured a key pair that does not contain the region name, they would face an error because the orchestrator always appends the region to the key pair name (even in the case of byo instance). Now that is handled.

This is handled with a bring your own instance test on a g5.24xl for llama3.1 8b instruct.