search

brainlid / langchain

Elixir implementation of an AI focused LangChain style framework.
https://hexdocs.pm/langchain/
Other
572 stars 68 forks source link

Support for Embeddings #40

Closed slashmili closed 10 months ago

slashmili commented 10 months ago

Hey there!

I'd like to see LangChain supports for embeddings! Probably first the OpenAPI's embeddings and then HuggingFace and others

Is this something that you have thought about and have a plan? I'd love to get some guidance and give it a try.

Disclaimer: I'm new to this field, if I say something that doesn't make sense please point out.

slashmili commented 10 months ago

After looking around it seems having something like ConversationalRetrievalChain and vectorstores are necessary to be able to use the embeddings in a meaningful way

brainlid commented 10 months ago

Hello @slashmili! I don't have any immediate plans in that direction. Yes, I believe that vectorstores are needed as well as some foundational pieces.

I'd love to have support for this for use with Agents or Assistants. However, I'm coming at it from a different direction and working on building more advanced Assistants (which is valuable alone) before focusing on the document embeddings.

Unfortunately, I can't offer much guidance in this area either. :slightly_smiling_face:

slashmili commented 10 months ago

Thanks for your feedback, I close this issue for now.

I'll try to fill my knowledge gap and come back to you again in few months 😃

cpursley commented 6 months ago

Is there any more interest in this? It would be neat to pair up with pgvector maybe alongside pgvector-elixir

Also, here's a neat talk on the subject: https://www.youtube.com/watch?v=RFnZB76KVWk

brainlid commented 6 months ago

@cpursley, I agree it would very cool and I would love to have it in. However, I don't personally have the bandwidth for it and I don't have a personal need for it at the moment.

I'm open to PRs!

slashmili commented 6 months ago

@brainlid is there a good place to have the discussion? e.g Elixir Slack, Elixir Forum, Elixir Discord server or here in the discussion section of GitHub repo. I have some idea I'd like to share and get some feedback before implementing anything.

cpursley commented 6 months ago

I'd also love a place do discuss langchain. My suggestion is a channel in the Elixir slack (I really don't like discord).

brainlid commented 6 months ago

I'd like to try using Github discussions for this! I haven't used them before, but it could help keep things all in one place. However, I did create the #langchain channel in the Elixir Slack as well. These are experiments and I may find one works better than the other. We'll see!

https://github.com/brainlid/langchain/discussions/91 - Embeddings discussion

@slashmili @cpursley