[Question]:With DynamicLLMPathExtractor，it shows “Request timed out”

[yuntong613]

Question Validation

• [X] I have searched both the documentation and discord for an answer.

Question

from llama_index.core import PropertyGraphIndex, PromptTemplate, SimpleDirectoryReader
from llama_index.core.indices.property_graph import SchemaLLMPathExtractor, DynamicLLMPathExtractor
from llama_index.core.prompts import PromptType
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.graph_stores.neo4j import Neo4jPropertyGraphStore
from llama_index.llms.openai_like import OpenAILike

graph_store = Neo4jPropertyGraphStore(
    username="neo4j",
    #    password="7q8j9k10",
    password="NEO4J@23.94",
    #    url="bolt://192.168.0.2:7687",
    url="bolt://10.10.23.94:7687",
)

llm_local = OpenAILike(model="llama-3.1-chinese", api_base="http://10.10.23.94:9997/v1", api_key="Not empty")

embed_model_m3e = OpenAIEmbedding(model_name="manual-moka-ai--m3e-base", api_base="http://10.10.23.94:9997/v1",
                                  api_key="Not empty")

MY_DEFAULT_DYNAMIC_EXTRACT_TMPL = (
    "从给定文本中提取最多 {max_knowledge_triplets} 个知识三元组。"
    "每个三元组的形式应为 (head, relation, tail)，包括各自的类型。\n"
    "---------------------\n"
    "初始本体论：\n"
    "实体类型：{allowed_entity_types}\n"
    "关系类型：{allowed_relation_types}\n"
    "\n"
    "使用这些类型作为起点，但如果根据上下文需要，可以引入新类型。\n"
    "\n"
    "指南：\n"
    "- 输出为 JSON 格式：[{{'head': '', 'head_type': '', 'relation': '', 'tail': '', 'tail_type': ''}}]\n"
    "- 使用最完整形式的实体（例如，用 'United States of America' 而不是 'USA'）\n"
    "- 保持实体简洁（最多 3-5 个词）\n"
    "- 将复杂短语拆分为多个三元组\n"
    "- 确保知识图谱连贯且易于理解\n"
    "---------------------\n"
    "示例：\n"
    "文本：Tim Cook, CEO of Apple Inc., announced the new Apple Watch that monitors heart health. "
    "UC Berkeley researchers studied the benefits of apples.\n"
    "输出：\n"
    "[{{'head': 'Tim Cook', 'head_type': 'PERSON', 'relation': 'CEO_OF', 'tail': 'Apple Inc.', 'tail_type': 'COMPANY'}},\n"
    " {{'head': 'Apple Inc.', 'head_type': 'COMPANY', 'relation': 'PRODUCES', 'tail': 'Apple Watch', 'tail_type': 'PRODUCT'}},\n"
    " {{'head': 'Apple Watch', 'head_type': 'PRODUCT', 'relation': 'MONITORS', 'tail': 'heart health', 'tail_type': 'HEALTH_METRIC'}},\n"
    " {{'head': 'UC Berkeley', 'head_type': 'UNIVERSITY', 'relation': 'STUDIES', 'tail': 'benefits of apples', 'tail_type': 'RESEARCH_TOPIC'}}]\n"
    "---------------------\n"
    "文本：{text}\n"
    "输出：\n"
)

MY_DEFAULT_DYNAMIC_EXTRACT_PROMPT = PromptTemplate(
    MY_DEFAULT_DYNAMIC_EXTRACT_TMPL, prompt_type=PromptType.KNOWLEDGE_TRIPLET_EXTRACT
)

kg_extractor = DynamicLLMPathExtractor(
    llm=llm_local,
    extract_prompt=MY_DEFAULT_DYNAMIC_EXTRACT_PROMPT,
    max_triplets_per_chunk=20,
    num_workers=4,
    # Let the LLM infer entities and their labels (types) on the fly
    allowed_entity_types=None,
    # Let the LLM infer relationships on the fly
    allowed_relation_types=None,
    # LLM will generate any entity properties, set `None` to skip property generation (will be faster without)
    allowed_relation_props=[],
    # LLM will generate any relation properties, set `None` to skip property generation (will be faster without)
    allowed_entity_props=[],
)

documents = SimpleDirectoryReader("./data/").load_data()
dynamic_index = PropertyGraphIndex.from_documents(
    documents,
    llm=llm_local,
    embed_kg_nodes=False,
    kg_extractors=[kg_extractor],
    show_progress=True,
)

graph_store.close()

[dosubot[bot]]

Hey @yuntong613! I'm here to help you with any bugs, questions, or contributions you may have. Let's work together to solve this issue!

To resolve "Request timed out" errors when using the DynamicLLMPathExtractor from the llama_index library, you can try the following steps:

1. Increase Timeout Settings: Ensure that the timeout settings for the language model (LLM) are sufficiently high to handle the requests. This might involve modifying the LLM's configuration if it supports timeout settings.

2. Reduce Load: Decrease the number of workers or the maximum number of paths per chunk to reduce the load on the LLM. This can be done by adjusting the num_workers and max_triplets_per_chunk parameters in the DynamicLLMPathExtractor initialization.

3. Retry Mechanism: Implement a retry mechanism to handle transient network issues or temporary unavailability of the LLM service.

Here is an example of how you can adjust the num_workers and max_triplets_per_chunk parameters:

kg_extractor = DynamicLLMPathExtractor(
    llm=llm_local,
    extract_prompt=MY_DEFAULT_DYNAMIC_EXTRACT_PROMPT,
    max_triplets_per_chunk=5,  # Reduce the number of triplets per chunk
    num_workers=2,  # Reduce the number of workers
    allowed_entity_types=None,
    allowed_relation_types=None,
    allowed_relation_props=[],
    allowed_entity_props=[],
)

Additionally, you can wrap the LLM call in a retry mechanism:

import asyncio
import aiohttp
from tenacity import retry, wait_fixed, stop_after_attempt

class DynamicLLMPathExtractor(TransformComponent):
    # ... existing code ...

    @retry(wait=wait_fixed(2), stop=stop_after_attempt(3))
    async def _aextract(self, node: BaseNode) -> BaseNode:
        """Extract triples from a node."""
        assert hasattr(node, "text")
        text = node.get_content(metadata_mode="llm")
        try:
            llm_response = await self.llm.apredict(
                self.extract_prompt,
                text=text,
                max_knowledge_triplets=self.max_triplets_per_chunk,
            )
            triples = self.parse_fn(llm_response)
        except (ValueError, aiohttp.ClientError) as e:
            print(f"Error: {e}, retrying...")
            raise
        except asyncio.TimeoutError:
            print("Request timed out, retrying...")
            raise

        # ... existing code to process triples ...

        return node

By implementing these changes, you can mitigate the "Request timed out" errors and improve the reliability of the DynamicLLMPathExtractor ^[1][2][3].

To continue talking to Dosu, mention @dosu.