Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.
154
stars
22
forks
source link
PR for lmi-dist + trt config files on p4, p5 #69
Closed
madhurprash closed 2 months ago
This PR contains the following config files:
mistral trt config files: p4d on instruct v1 & v2, p5 on instruct v2
mistral lmi-dist config files: p4d on instruct v2, p5 on instruct v2