search

lm-sys / RouteLLM

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality!
Apache License 2.0
3.3k stars 250 forks source link

Optimization for MF #17

Open thwu1 opened 4 months ago

thwu1 commented 4 months ago

9 I added precompution for collapsing three linear transformations without modifying the loading part. @iojw Do you want to do a speed test?