issues
search
vectorch-ai
/
ScaleLLM
A high-performance inference system for large language models, designed for production environments.
https://docs.vectorch.com/
Apache License 2.0
316
stars
23
forks
source link
feat: added openai compatible logprobs support
#232
Closed
guocuimi
closed
3 weeks ago
guocuimi
commented
3 weeks ago
[ ] add logprobs support for grpc server
[ ] add logprobs support for speculative decoding