issues
search
neuralmagic
/
deepsparse
Sparsity-aware deep learning inference runtime for CPUs
https://neuralmagic.com/deepsparse/
Other
2.94k
stars
169
forks
source link
[TextGeneration] add a test to evaluate text generation with continuous batching
#1564
Closed
dsikka
closed
5 months ago
dsikka
commented
5 months ago
Summary
Quick test which was missing
Once the new logger is integrated with the pipeline, will need to update to add a test which makes sure we actually batch given sufficient load
Summary