search

vasilecampeanu / obsidian-weaver

Weaver is an Obsidian plugin that integrates a ChatGPT-like interface into your note-taking workflow. This plugin makes it easy to access AI-generated suggestions and insights within Obsidian, helping you improve your writing and brainstorming process.
MIT License
202 stars 8 forks source link

Add local language models support #42

Closed kabachuha closed 1 month ago

kabachuha commented 1 year ago

Hi, @vasilecampeanu !

While GPT-* models are awesome, there's an issue of privacy arising from using an external third party API, especially when the information may be sensitive. More than that, they don't allow as much customization for one's needs as the local models. In my mind, adding support of local models is one of the top priorities of such a plugin.

Fortunately, thanks for people like oobabooga making a wrapper for many possible local models like LLaMa, Vicuna, MPT, Galactica, etc it's pretty straightforward to implement. Just replace the call for OpenAI's API to oobabooga API's endpoint. For example, how I did this https://github.com/rizerphe/obsidian-companion/pull/13 (may require some tweaks)

supports streaming mode as well

vasilecampeanu commented 1 year ago

Hello, I'm sorry for the late response. Thank you for the pull request and for bringing up the issue. I intend to add support for local models, but I'm quite busy at the moment and haven't found the time to delve into it yet. Currently, my primary focus is to release the plugin to the community list.