Fix the logic for perplexity evaluation (`Not enough kv_cache capacity to run generation. Please use a larger sequence_length or a shorter prompt`) - Githubissues

neuralmagic / deepsparse

Sparsity-aware deep learning inference runtime for CPUs

https://neuralmagic.com/deepsparse/

Other

3.03k stars 173 forks source link

Fix the logic for perplexity evaluation (`Not enough kv_cache capacity to run generation. Please use a larger sequence_length or a shorter prompt`) #1633

Closed dbogunowicz closed 7 months ago

dbogunowicz commented 7 months ago

Closing, due to inactivity.