search

weaviate / Verba

Retrieval Augmented Generation (RAG) chatbot powered by Weaviate
BSD 3-Clause "New" or "Revised" License
6k stars 639 forks source link

Using a custom embedder #119

Closed israaexol closed 1 week ago

israaexol commented 6 months ago

Hello, I would like to use a custom embedding model hosted via the HuggingFace library other than the ones already available via the Verba interface (Mini-LM, Ada Embedder, and Cohere Embedder), but I can't seem to find a way to do that. I tried connecting a Weaviate cluster but I don't know how I can add custom embedders to it as well.

I have the same inquiry about LLMs available via the HuggingFace interface and the possibility of using other ones in Verba other than LLAMA.

Could you please provide me with some guidance regarding this matter? I'd be most thankful!

Thank you in advance.

moncefarajdal commented 5 months ago

I'm also looking for this feature. I tried to hack into the Llama generator to use falcon-7b model from HuggingFace, but things seem not to be working. If anyone has already done it please let us know! Thanks guys.

bakongi commented 5 months ago

Maybe this help https://github.com/weaviate/Verba/issues/128#issue-2216314195

thomashacker commented 5 months ago

Great point! It would be a good idea to add some Resources for custom components!

Mr-Jack-Tung commented 5 months ago

I try this way:

File: manager.py

... from transformers import AutoTokenizer encoding = AutoTokenizer.from_pretrained("sentence-transformers/all-MiniLM-L6-v2") "# encoding = tiktoken.encoding_for_model("gpt-3.5-turbo") # replace this code" ... item_tokens = encoding.encode(item_dict["content"], add_special_tokens=False) ... "# If adding the entire new item exceeds the max tokens" if accumulated_tokens + len(item_tokens) > max_tokens: "# Calculate how many tokens we can add from this item" remaining_space = max_tokens - accumulated_tokens truncated_content = encoding.decode(item_tokens[:remaining_space]) ...

encoding only using for caculate how many tokens we can add, so why must using ChatGPT for this task ?! ^^ ... that's why I change this code to using AutoTokenizer.from_pretrained("sentence-transformers/all-MiniLM-L6-v2") free for this simple task ^^

thomashacker commented 5 months ago

Good point! It is definitely something that can be improved! Your code will be helpful, thanks

thomashacker commented 1 week ago

Closing this for now