Closed FairyHan closed 3 weeks ago
This is a known bug. The root cause is due to exceeding the TPM/RPM of the LLM too many times. Please set the tokens_per_minute
and requests_per_minute
config setting to an appropriate value other than 0
. When set to 0, graphrag will run unthresholded. graphrag will make retry attempts, but eventually it will exceed the max retry limit. This causes a corruption in the underlying dataframe that is passed around inside graphrag.
We are looking at better error handling to improve this behavior.
This is a known bug. The root cause is due to exceeding the TPM/RPM of the LLM too many times. Please set the
tokens_per_minute
andrequests_per_minute
config setting to an appropriate value other than0
. When set to 0, graphrag will run unthresholded. graphrag will make retry attempts, but eventually it will exceed the max retry limit. This causes a corruption in the underlying dataframe that is passed around inside graphrag.We are looking at better error handling to improve this behavior.
@jgbradley1 hi,Why is it that when I change the tokens_per_minute and requests_per_minute values in settings.yaml, but I see that the llm configuration item is not changed to the corresponding value in the log? Only the configuration item in embeddings llm can be modified successfully.
Can you use the proxy http://172.21.51.30:3128 normally? I checked the source code of the latest released version and it seems that the proxy parameter of openaiclient is not passed, and proxy cannot be used under LLM but api_proxy instead. I had to modify it to make it work.
I have a similar problem when running the indexing job
It continuously retries after hitting TMP threshold
The index job is stuck at 6.25% completion
Pipeline configuration : lm: type: azure_openai_chat api_base: $GRAPHRAG_API_BASE api_version: $GRAPHRAG_API_VERSION model: $GRAPHRAG_LLM_MODEL deployment_name: $GRAPHRAG_LLM_DEPLOYMENT_NAME cognitive_services_endpoint: $GRAPHRAG_COGNITIVE_SERVICES_ENDPOINT model_supports_json: True tokens_per_minute: 8000 requests_per_minute: 1 thread_count: 15 concurrent_requests: 1
parallelization: stagger: 0.75 num_threads: 2
async_mode: threaded
embeddings: async_mode: threaded llm: type: azure_openai_embedding api_base: $GRAPHRAG_API_BASE api_version: $GRAPHRAG_API_VERSION batch_size: 16 model: $GRAPHRAG_EMBEDDING_MODEL deployment_name: $GRAPHRAG_EMBEDDING_DEPLOYMENT_NAME cognitive_services_endpoint: $GRAPHRAG_COGNITIVE_SERVICES_ENDPOINT tokens_per_minute: 100000 concurrent_requests: 15 requests_per_minute: 1000 thread_count: 25 max_retries: 50 parallelization: stagger: 0.75 num_threads: 10 vector_store: type: azure_ai_search collection_name: PLACEHOLDER title_column: name overwrite: True url: $AI_SEARCH_URL audience: $AI_SEARCH_AUDIENCE
This issue has been closed after being marked as stale for five days. Please reopen if needed.
Describe the issue
我在利用密钥和代理使用openai官方模型的过程中出错了
Steps to reproduce
使用代理proxy访问openai官方模型,使用个人密钥,其余设置保持默认
GraphRAG Config Used
Logs and screenshots
Additional Information