search

zed-industries / zed

Code at the speed of thought – Zed is a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.
https://zed.dev
Other
47.89k stars 2.81k forks source link

User-Configurable LLM Api settings at Runtime #13400

Open carnivoroustoad opened 3 months ago

carnivoroustoad commented 3 months ago

Check for existing issues

Describe the feature

Right now, we have to wait for stable releases or compile the nightly build ourselves to use the latest language models w/ the API, as the configurations are preset. Case in point: to use Claude Sonnet 3.5 just yesterday, I had to build the nightly version. That's a pain.

Given the speed of LLM advancement right now, this isn't optimal. What we really need is a layer of abstraction that lets us add our own custom API configurations at runtime. This would allow us to quickly integrate new models from any provider without being at the mercy of new releases or having to dive into building the editor ourselves. Don't see any reason why this shouldn't be possible looking through the source.

If applicable, add mockups / screenshots to help present your vision of the feature

No response

amtoaer commented 3 months ago

I attempted to implement this kind of configuration for OpenAI-compatible services:

If that PR is approved, other LLM providers could adopt the same approach.