Closed wsxiaoys closed 1 day ago
Taking a look at this, just fyi to anyone else looking at this issue. Will update with PR.
Hi @wsxiaoys, this is my first time coding in rust. I am well versed in c++ so I don't anticipate any major issues, but I wanted to check with you to see how you anticipated this issue being solved. Was there a specific implementation that you envisioned? Thanks!
Releasing in 0.16 - If you're interested, you might test it earlier at https://github.com/TabbyML/tabby/releases/tag/v0.16.0-rc.1.
Released in 0.16
Please describe the feature you want
Related: https://github.com/TabbyML/tabby/issues/2652
This allows a local deployment could use fewer vram / computing in local setup
Additional context Code location:
Please reply with a 👍 if you want this feature.