neuralmagic / deepsparse

Sparsity-aware deep learning inference runtime for CPUs

https://neuralmagic.com/deepsparse/

Other

2.94k stars 169 forks source link

Closed dsikka closed 5 months ago

dsikka commented 5 months ago

Summary

To address this ticket: https://app.asana.com/0/1205229323407165/1206459001098561
Also update condition for kv cache capacity check to prevent repeat check