search

rizerphe / obsidian-companion

Autocomplete your obsidian notes with AI, including ChatGPT, through a copilot-like interface.
MIT License
189 stars 23 forks source link

Add oobabooga text generation webui api completer #13

Closed kabachuha closed 1 year ago

kabachuha commented 1 year ago

Connects to local models: LLaMa (including llama.cpp), local GPT-derivatives via oobabooga's text generation webui

Closes #11

rizerphe commented 1 year ago

Just opened this; will see what I can do in the meantime

kabachuha commented 1 year ago

Yes, thanks! @rizerphe . It was a real problem

rizerphe commented 1 year ago

Just opened this as well; currently working under the assumption that my changes get accepted

rizerphe commented 1 year ago

Overall this seems to be working for me; I hope the generation web UI gets CORS support; other than that the only real problem left seems to be model switching - I don't like the idea of it in general, and I don't like how clumsily I had to do it even more.

Oobabooga only allows you to have one model loaded at a time, and switching takes a lot of time, and this isn't too convenient. For now, what I do is before every generate call, I check which model is loaded (through another API call) and load the one I need if necessary. Doing it differently could be smarter but requires a bit of a rewrite.

rizerphe commented 1 year ago

Quickly wanted to mention that oobabooga merged my pull request, therefore you don't need to use my fork anymore.