search

brainlid / langchain

Elixir implementation of a LangChain style framework.
https://hexdocs.pm/langchain/
Other
503 stars 58 forks source link

Add support for Bumblebee functions? #88

Open brainlid opened 3 months ago

brainlid commented 3 months ago

Bumblebee doesn't support a constrained output of only valid JSON.

In the early days of LangChain, they implemented an alternate approach for functions that was more of a hack, but worked well enough.

Investigate if this approach could work for bringing functions to Bumblebee models. It would still help if the model being run understood JSON, functions, etc.

hwuethrich commented 2 months ago

Hi @brainlid

I'd be very interested in this feature. Especially now that Mistral released their Mixtral-8x22B-Instruct-v0.1 model, which supports function calling! 🎉

Here is an example for the expected prompt template: https://github.com/mistralai/mistral-common/blob/main/examples/tokenizer.ipynb

My company (@zebbra) might have some capacity to make this contribution in case you are interested? I assume it is just a matter of adding a new clause to https://github.com/brainlid/langchain/blob/main/lib/utils/chat_templates.ex, right? Or are there any other pieces missing from the puzzle?

Anyway, thanks for this library (and podcast! 🙂)

Edit: I guess ChatBumblebee would have to be able to parse the function calls as well

hwuethrich commented 2 months ago

Probably not as easy as I thought, because Mixtral is currently no supported by Bumblebee: https://github.com/elixir-nx/bumblebee/issues/365