Can I use a Custom embedder in the crewai_tools?

[rohanprasadsap]

I am trying to use the the crewai_tools, the RAG tools in particular. Lets take the DirectorySearchTool() in this case.

How do I use a custom embedder provider? The only options that embed chain have are from the following list. 'openai', 'gpt4all', 'huggingface', 'vertexai', 'azure_openai', 'google', 'mistralai', 'nvidia' Embedding models for which I need API keys.

directory_search_tool = DirectorySearchTool(
    config=dict(
        llm=dict(
            provider="ollama", # Options include ollama, google, anthropic, llama2, and more
            config=dict(
                model="llama2",
                # Additional configurations here
            ),
        ),
        embedder=dict(
            provider="google", # or openai, ollama, ...
            config=dict(
                model="models/embedding-001",
        ),
    )
)

I know that CrewAI uses embedchain module for RAG operations and the default provider is set to openai. And to use openai embedding model one needs to set OPENAI_API_KEY as environment variable.

But the situation is that the LLMs that I am using is from the from my Company's/Organization's langchain.openai import ChatOpenAI module (A custom module basically) and I don't need to set API keys and stuff for using the LLMs. Something like this:

llm = ChatOpenAI(proxy_model_name='gpt-4-32k')

which is of type: .proxy.langchain.openai.ChatOpenAI

Question: How do I use a custom embedder provider? Is there a provider called "custom" or anything that i can use in my case?

[breakerh]

You can use lm-studio or any other provider that uses openai api format.

embedder=dict(
        provider="openai",
        config=dict(
            model="CompendiumLabs/bge-large-en-v1.5-gguf",
            api_key="lm-studio",
            api_base="http://localhost:1234/v1"
        ),
    ),

[cbruyndoncx]

Not in current version v0.30.11 with ollama, see other post.

[hbwliao]

this one: https://github.com/xorbitsai/inference

[chaofanat]

您可以使用lm-studio或任何其他使用 openai api 格式的提供商。

embedder=dict(
        provider="openai",
        config=dict(
            model="CompendiumLabs/bge-large-en-v1.5-gguf",
            api_key="lm-studio",
            api_base="http://localhost:1234/v1"
        ),
    ),

oh,that's not very good all in open-AI

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.