search

lm-sys / RouteLLM

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality!
Apache License 2.0
2.78k stars 204 forks source link

Optimization for MF #17

Open thwu1 opened 1 month ago

thwu1 commented 1 month ago

9 I added precompution for collapsing three linear transformations without modifying the loading part. @iojw Do you want to do a speed test?