search

manubot / manubot-ai-editor

BSD 3-Clause "New" or "Revised" License
37 stars 8 forks source link

Ability to use Python LLM package plugins #51

Open castedo opened 1 month ago

castedo commented 1 month ago

This is a feature idea which I might add to copyaid.it. Might be something manubot-ai-editor wants to do too.

The llm Python package has a large number of plugins that can be used to easily (I assume) switch between different LLM APIs and local LLM models:

https://llm.datasette.io/en/stable/plugins/directory.html#plugin-directory

I imagine this plugin API interface is simple enough that it's a quick way to expand out into many different LLM implementations.

miltondp commented 1 month ago

Thank you, @castedo. We are currently implementing this feature but with LangChain. We haven't compared it with this package. Do you have any thoughts about this?

castedo commented 1 month ago

LangChain does much more than just make it easy to switch between LLM sources (i.e., OpenAI vs Anthropic vs local). Whereas calling the llm package plugins via whatever internal interface is used by llm is just one possible easy solution.

If you are using LangChain anyway for other reasons, then ease of LLM source switching is a nice secondary benefit that pops out and there is little reason to use the llm package interface.

But if all you want to achieve is ease of LLM source switching then LangChain is way overkill and will pull in a ton of dependencies (I think). llm is just a simple CLI tool kind of similar copyaid and manubot-ui-editor in CLI form.