Openai-compatible api support

[Dormiveglia-elf]

I think it is necessary to add an example to illustrate how to use openai-compatible api like deepseek to use fast-graphrag, since the official openai api is too expensive.

For example, in Lightrag, they provide this https://github.com/HKUDS/LightRAG/blob/main/examples/lightrag_openai_compatible_demo.py

[liukidar]

Hello, we have added an example folder. Let me know if that solves the issue :)

[Dormiveglia-elf]

Hello, we have added an example folder. Let me know if that solves the issue :)

Not working, I guess some libraries may still require the environment variable OPENAI_API_KEY to be set, even if the API key is already provided in the code. Error below: Traceback (most recent call last): File "/root/fast-graphrag/examples/custom_llm.py", line 32, in config=GraphRAG.Config( File "", line 10, in init File "/usr/local/lib/python3.10/dist-packages/fast_graphrag/init.py", line 69, in DefaultVectorStorageConfig(embedding_dim=DefaultEmbeddingService().embedding_dim) File "", line 8, in init File "/usr/local/lib/python3.10/dist-packages/fast_graphrag/_llm/_llm_openai.py", line 115, in post_init__ self.embedding_async_client: AsyncOpenAI = AsyncOpenAI(api_key=self.api_key) File "/usr/local/lib/python3.10/dist-packages/openai/_client.py", line 319, in init__ raise OpenAIError( openai.OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable

[Dormiveglia-elf]

@liukidar

[liukidar]

Can you try to use the last commit and see if it works? On my machine, the example in custom_llm.py works both by either setting OPEN_API_KEY as an env variable or by passing api_key to the constructor of OpenAILLMService and OpenAIEmbeddingService

If it doesn't, can you share the code you're using (you can just put a random api key since it is just necessary to start the code).

[Dormiveglia-elf]

Can you try to use the last commit and see if it works? On my machine, the example in custom_llm.py works both by either setting OPEN_API_KEY as an env variable or by passing api_key to the constructor of OpenAILLMService and OpenAIEmbeddingService

If it doesn't, can you share the code you're using (you can just put a random api key since it is just necessary to start the code).

Still not working, below is my code and error is the same as my above comment.

"""Example usage of GraphRAG with custom LLM and Embedding services compatible with the OpenAI API."""
from typing import List

from dotenv import load_dotenv

from fast_graphrag import GraphRAG
from fast_graphrag._llm import OpenAIEmbeddingService, OpenAILLMService

load_dotenv()

DOMAIN = "Bitcoin and Cryptocurrency"
QUERIES: List[str] = [
    "What is the double-spending problem, and how does Bitcoin address it?",
    "How does Bitcoin ensure transaction security without a central authority?",
    "Explain the role of proof-of-work in Bitcoin’s network.",
    "How does Bitcoin's timestamp server contribute to transaction verification?",
    "What are the steps involved in the Bitcoin network’s transaction process?",
    "Describe the incentive mechanism for Bitcoin miners.",
    "How does Bitcoin's blockchain use the Merkle Tree structure?",
    "What is simplified payment verification in Bitcoin, and how does it work?",
    "How does Bitcoin maintain privacy for its users?",
    "What is the probability of a successful attack on the Bitcoin network, and how is it calculated?"
]
ENTITY_TYPES: List[str] = ["Bitcoin", "Cryptocurrency", "Blockchain", "Event", "Technology", "Person", "Organization", "Time", "Location", "Contract"]

working_dir = "./test_bitcoin"
grag = GraphRAG(
    working_dir=working_dir,
    domain=DOMAIN,
    example_queries="\n".join(QUERIES),
    entity_types=ENTITY_TYPES,
    config=GraphRAG.Config(
        llm_service=OpenAILLMService(model="deepseek-chat", base_url="https://api.deepseek.com/", api_key=""),
        embedding_service=OpenAIEmbeddingService(
            model="text-embedding-3-small",
            base_url="",
            api_key="",
            embedding_dim=1536,  # the output embedding dim, the embedding dim of vector storage must match this value
        ),
    ),
)

with open("./bitcoin.txt") as f:
    grag.insert(f.read())

print(grag.query("Introduce Bitcoin briefly").response)

[liukidar]

I'm sorry but I'm really unable to replicate your issue, the code you sent works perfectly fine on my machine. Can you check you're using the last commit of this repo?