search

microsoft / graphrag

A modular graph-based Retrieval-Augmented Generation (RAG) system
https://microsoft.github.io/graphrag/
MIT License
18.97k stars 1.86k forks source link

[Issue]: <title> {"type": "error", "data": "Error Invoking LLM", "stack": #1228

Closed dipakmeher closed 1 month ago

dipakmeher commented 1 month ago

Do you need to file an issue?

Describe the issue

I am getting an error that says 'Error Invoking LLM' in my code. I’ve tried a few tweaks, but nothing has worked. Any help with this would be appreciated.

Error: [Issue]: {"type": "error", "data": "Error Invoking LLM", "stack": : "Traceback (most recent call last):\n File \"/scratch/dmeher/custom_env/miniforge/envs/graphrag_env/lib/python3.10/site-packages/httpx/_transports/default.py\", line 72, in map_httpcore_exceptions\n yield\n File \"/scratch/dmeher/custom_env/miniforge/envs/graphrag_env/lib/python3.10/site-packages/httpx/_transports/default.py\", line 377, in handle_async_request\n resp = await self._pool.handle_async_request(req)\n File \"/scratch/dmeher/custom_env/miniforge/envs/graphrag_env/lib/python3.10/site-packages/httpcore/_async/connection_pool.py\", line 216, in handle_async_request\n raise exc from None\n File </p> <h3>Steps to reproduce</h3> <p>You can replicate this issue by using mistral as llm from ollama.</p> <h3>GraphRAG Config Used</h3> <pre><code class="language-yaml"># Paste your config here encoding_model: cl100k_base skip_workflows: [] llm: api_key: ${GRAPHRAG_API_KEY} type: openai_chat # or azure_openai_chat #model: gpt-4-turbo-preview model: mistral model_supports_json: true # recommended if this is available for your model. #max_tokens: 4000 # request_timeout: 180.0 api_base: http://localhost:11434/v1 #https://<instance>.openai.azure.com # api_version: 2024-02-15-preview # organization: <organization_id> # deployment_name: <azure_model_deployment_name> # tokens_per_minute: 150_000 # set a leaky bucket throttle # requests_per_minute: 10_000 # set a leaky bucket throttle #max_retries: 1 # max_retry_wait: 10.0 # sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times #concurrent_requests: 1 # the number of parallel inflight requests that may be made # temperature: 0 # temperature for sampling # top_p: 1 # top-p sampling # n: 1 # Number of completions to generate parallelization: stagger: 0.3 # num_threads: 50 # the number of threads to use for parallel processing async_mode: threaded # or asyncio embeddings: ## parallelization: override the global parallelization settings for embeddings async_mode: threaded # or asyncio # target: required # or all # batch_size: 16 # the number of documents to send in a single request # batch_max_tokens: 8191 # the maximum number of tokens to send in a single request llm: api_key: ${GRAPHRAG_API_KEY} type: openai_embedding # or azure_openai_embedding #model: text-embedding-3-small model: nomic_embed_text api_base: http://localhost:11434/api #https://<instance>.openai.azure.com # api_version: 2024-02-15-preview # organization: <organization_id> # deployment_name: <azure_model_deployment_name> # tokens_per_minute: 150_000 # set a leaky bucket throttle # requests_per_minute: 10_000 # set a leaky bucket throttle # max_retries: 10 # max_retry_wait: 10.0 # sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times # concurrent_requests: 25 # the number of parallel inflight requests that may be made chunks: size: 1200 overlap: 100 group_by_columns: [id] # by default, we don't allow chunks to cross documents input: type: file # or blob file_type: text # or csv base_dir: "input" file_encoding: utf-8 file_pattern: ".*\\.txt$" cache: type: file # or blob base_dir: "cache" # connection_string: <azure_blob_storage_connection_string> # container_name: <azure_blob_storage_container_name> storage: type: file # or blob base_dir: "output/${timestamp}/artifacts" # connection_string: <azure_blob_storage_connection_string> # container_name: <azure_blob_storage_container_name> reporting: type: file # or console, blob base_dir: "output/${timestamp}/reports" # connection_string: <azure_blob_storage_connection_string> # container_name: <azure_blob_storage_container_name> entity_extraction: ## strategy: fully override the entity extraction strategy. ## type: one of graph_intelligence, graph_intelligence_json and nltk ## llm: override the global llm settings for this task ## parallelization: override the global parallelization settings for this task ## async_mode: override the global async_mode settings for this task prompt: "prompts/entity_extraction.txt" entity_types: [organization,person,geo,event] max_gleanings: 1 summarize_descriptions: ## llm: override the global llm settings for this task ## parallelization: override the global parallelization settings for this task ## async_mode: override the global async_mode settings for this task prompt: "prompts/summarize_descriptions.txt" max_length: 500 claim_extraction: ## llm: override the global llm settings for this task ## parallelization: override the global parallelization settings for this task ## async_mode: override the global async_mode settings for this task # enabled: true prompt: "prompts/claim_extraction.txt" description: "Any claims or facts that could be relevant to information discovery." max_gleanings: 1 community_reports: ## llm: override the global llm settings for this task ## parallelization: override the global parallelization settings for this task ## async_mode: override the global async_mode settings for this task prompt: "prompts/community_report.txt" max_length: 2000 max_input_length: 8000 cluster_graph: max_cluster_size: 10 embed_graph: enabled: false # if true, will generate node2vec embeddings for nodes # num_walks: 10 # walk_length: 40 # window_size: 2 # iterations: 3 # random_seed: 597832 umap: enabled: false # if true, will generate UMAP embeddings for nodes snapshots: graphml: false raw_entities: false top_level_nodes: false local_search: # text_unit_prop: 0.5 # community_prop: 0.1 # conversation_history_max_turns: 5 # top_k_mapped_entities: 10 # top_k_relationships: 10 # llm_temperature: 0 # temperature for sampling # llm_top_p: 1 # top-p sampling # llm_n: 1 # Number of completions to generate # max_tokens: 12000 global_search: # llm_temperature: 0 # temperature for sampling # llm_top_p: 1 # top-p sampling # llm_n: 1 # Number of completions to generate # max_tokens: 12000 # data_max_tokens: 12000 # map_max_tokens: 1000 # reduce_max_tokens: 2000 # concurrency: 32 </code></pre> <h3>Logs and screenshots</h3> <p><img referrerpolicy="no-referrer" src="https://github.com/user-attachments/assets/92fdf5e5-b930-4499-8859-33c8f379a4f9" alt="Screenshot 2024-09-29 at 10 43 30 PM" /></p> <h3>Additional Information</h3> <ul> <li>GraphRAG Version:</li> <li>Operating System: Linux</li> <li>Python Version: 3.10</li> <li>Related Issues:</li> </ul> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/dipakmeher"><img src="https://avatars.githubusercontent.com/u/26577810?v=4" />dipakmeher</a> commented <strong> 1 month ago</strong> </div> <div class="markdown-body"> <p>Issue resolved: The problem was with my input, which I believe was too long for the LLM models I was using: Mistral and Nomic Embed Text. It was a book from Gutenberg, and the code ran perfectly fine with OpenAI LLM models (for both entity extraction and embeddings) but not with Mistral and Nomic Embed Text. I shortened my input to test it, and that resolved the error.</p> <p>Currently, I am facing issue #1234. Any help with this would be appreciated.</p> </div> </div> <div class="page-bar-simple"> </div> <div class="footer"> <ul class="body"> <li>© <script> document.write(new Date().getFullYear()) </script> Githubissues.</li> <li>Githubissues is a development platform for aggregating issues.</li> </ul> </div> <script src="https://cdn.jsdelivr.net/npm/jquery@3.5.1/dist/jquery.min.js"></script> <script src="/githubissues/assets/js.js"></script> <script src="/githubissues/assets/markdown.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/highlight.min.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/languages/go.min.js"></script> <script> hljs.highlightAll(); </script> </body> </html>