James-QiuHaoran / LLM-serving-with-proxy-models

Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction
Apache License 2.0
16 stars 5 forks source link

Add per-class prediction error analysis #6

Closed James-QiuHaoran closed 3 months ago

James-QiuHaoran commented 3 months ago

See https://github.com/James-QiuHaoran/LLM-serving-with-proxy-models/commit/38c4a0f602677b2001222fda2e4ac696e539205b