search

huggingface / optimum-neuron

Easy, fast and very cheap training and inference on AWS Trainium and Inferentia chips.
Apache License 2.0
176 stars 51 forks source link

Improve tgi env wrapper for neuron #589

Closed oOraph closed 1 month ago

oOraph commented 2 months ago

when several compatible entries are found, take the one that uses the most cores and then has the biggest batch size