search

karthink / gptel

A simple LLM client for Emacs
GNU General Public License v3.0
1.66k stars 155 forks source link

Support for local / selfhosted GPTs? #63

Closed LazerJesus closed 1 year ago

LazerJesus commented 1 year ago

Hey, I love your package and use it every day. Especially since it is so universal (org, code, buffer, chat, shell, whatever) and integrated right into my os. which is something that Microsoft etc are only now announcing. emacs was first ;)

Now, to the subject at hand: With the dawn of open source coding assistants making it ever more feasible to run these networks either locally or on a self-provisioned cloud box, it would be amazing to move past vendor lock-in to "Open"AI towards a more flexible "plugin"-esque system.

Do you see a future where this package is more like a general interface for various GPTs?

karthink commented 1 year ago

Do you see a future where this package is more like a general interface for various GPTs?

@FinnFrotscher That is indeed the plan! Like many Emacs users, I have my reservations about depending on and supplying OpenAI with more data. I would also prefer not to depend on the cloud for AI completion.

However I'm not in a position to ride the bleeding edge of this technology -- I don't have the time to run and troubleshoot various local LLMs, nor do I have the GPU hardware for it. So progress in this direction will be slow. I'm open to PRs adding support for more models.

Especially since it is so universal (org, code, buffer, chat, shell, whatever) and integrated right into my os.

Glad to hear it. This is why I wrote a client instead of using one of the prescriptivist alternatives. In fact, your description is very concise, I'm going to use a version of this in the Readme -- thanks!

LazerJesus commented 1 year ago

glad to have provided my first contribution to this package haha. and I am glad to hear that this is indeed where the project is headed. since something like Starcoders still takes a massive GPU setup for inference, it will probably be a while before this is a mainstream need. I can't wait for the day tho. In combination with something like Tree of Thought agents, this will be insanely powerful. Especially in the environment that emacs provides, where everything is a function (which is just text at the end of the day) and introspectable.

LazerJesus commented 1 year ago

Hi @karthink i've summarized my ideas for AI+Emacs in a blogpost.

karthink commented 1 year ago

I've added preliminary support for local LLMs in #114. The discussion can continue there, so I'm closing this issue.