search

codelion / optillm

Optimizing inference proxy for LLMs
Apache License 2.0
1.64k stars 130 forks source link

Fix entropy decoding in local server #95

Closed codelion closed 1 week ago

codelion commented 1 week ago