search

James-QiuHaoran / LLM-serving-with-proxy-models

Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction | A tiny model can tell you the verbosity of an LLM (with low latency!)
Apache License 2.0
22 stars 5 forks source link

issues

Newest
Newest Most commented Recently updated Oldest Least commented Least recently updated