circlemind-ai / fast-graphrag

RAG that intelligently adapts to your use case, data, and queries
MIT License
1.98k stars 84 forks source link

采用vllm部署大模型,使用xinference部署的embedding,集成在代码中,出现了问题 #29

Open Joker-sad opened 13 hours ago

Joker-sad commented 13 hours ago

from fast_graphrag import GraphRAG from fast_graphrag._llm import OpenAIEmbeddingService, OpenAILLMService

from typing import List

DOMAIN = "Novel Analysis" QUERIES: List[str] = [ "Who are the main characters in this novel?", "What locations are featured in the story?", "What are the relationships between the characters?", ] ENTITY_TYPES: List[str] = ["Character", "Location", "Event"]

设置工作目录

working_dir = "./examples/ignore/hp"

初始化 GraphRAG,去掉不支持的 timeout 参数

grag = GraphRAG( working_dir=working_dir, domain=DOMAIN, example_queries="\n".join(QUERIES), entity_types=ENTITY_TYPES, config=GraphRAG.Config( llm_service=OpenAILLMService( model="XXXXX/Qwen2-7B-Instruct-GPTQ-Int4", base_url="http://XXXX:XXXX/v1/", api_key="token-12", ), embedding_service=OpenAIEmbeddingService( model="bge-base-zh", base_url="http://XXXXX:XXXX/v1/", api_key="token-12", embedding_dim=512,), ), )

try: with open("./book.txt", "r", encoding="utf-8") as f: story_content = f.read() grag.insert(story_content) except Exception as e: print(f"插入文本时发生错误: {e}")

查询示例

try: response = grag.query("这本小说的主要人物有哪些?").response print(response) except Exception as e: print(f"查询时发生错误: {e}")

报错:Extracting data: 0%| | 0/1 [00:00<?, ?it/sError during information extraction from document: Request timed out. Extracting data: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [27:13<00:00, 1633.24s/it] Error during query: Request timed out. 查询时发生错误: Request timed out.

问题是请求时间过程,但是我绝对实际的问题可能不是,总觉得是不是我部署的模型不能够在代码中使用