search

aws-samples / foundation-model-benchmarking-tool

Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.
MIT No Attribution
151 stars 21 forks source link

adding EC2 djl rest prediction support for llama3 on fmbench #113

Closed madhurprash closed 4 weeks ago

madhurprash commented 4 weeks ago

This PR contains:

  1. updated pricing.yml file with the ec2 g5.2xl pricing addition
  2. ec2_djl_predictor to handle post requests to the ep url
  3. new llama3 on ec2 byoe (more like bring your endpoint url :) ) config file