[PyTorch] DBRX model inference with Continuous batching

aws-neuron / aws-neuron-sdk

Powering AWS purpose-built machine learning chips. Blazing fast and cost effective, natively integrated into PyTorch and TensorFlow and integrated with your favorite AWS services

https://aws.amazon.com/machine-learning/neuron/

Other

464 stars 154 forks source link

[PyTorch] DBRX model inference with Continuous batching #963

Open eshalakhotia opened 3 months ago