search

aws-samples / foundation-model-benchmarking-tool

Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.
MIT No Attribution
151 stars 21 forks source link

EKS integration with FMBench #126

Closed madhurprash closed 1 week ago

madhurprash commented 2 weeks ago

This PR contains the following additions + changes:

  1. Config files for llama3 and mistral for deploying on the eks cluster
  2. eks manifest files for mistral and llama3
  3. scripts for eks_deploy and predict functions
  4. handling for eks endpoint deployments (changes in notebook 1, 3)
  5. Clean up code added for eks in notebook 5
  6. readme updates