search

microsoft / graphrag

A modular graph-based Retrieval-Augmented Generation (RAG) system
https://microsoft.github.io/graphrag/
MIT License
16.83k stars 1.58k forks source link

[running error] how to use openai ? #594

Closed FairyHan closed 3 weeks ago

FairyHan commented 1 month ago

Describe the issue

我在利用密钥和代理使用openai官方模型的过程中出错了

Steps to reproduce

使用代理proxy访问openai官方模型，使用个人密钥，其余设置保持默认

GraphRAG Config Used

encoding_model: cl100k_base
skip_workflows: []
llm:
  api_key: sk-ySJkHe4C。。。。
  type: openai_chat # or azure_openai_chat
  model: gpt-4-turbo-preview
  model_supports_json: true # recommended if this is available for your model.
  proxy: http://172.21.51.30:3128
  # max_tokens: 4000
  # request_timeout: 180.0
  # api_base: https://api.172.21.51.30:3128.com
  # api_version: 2024-02-15-preview
  # organization: <organization_id>
  # deployment_name: <azure_model_deployment_name>
  # tokens_per_minute: 150_000 # set a leaky bucket throttle
  # requests_per_minute: 10_000 # set a leaky bucket throttle
  # max_retries: 10
  # max_retry_wait: 10.0
  # sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
  # concurrent_requests: 25 # the number of parallel inflight requests that may be made

parallelization:
  stagger: 0.3
 # num_threads: 50 # the number of threads to use for parallel processing

async_mode: threaded # or asyncio

embeddings:
  ## parallelization: override the global parallelization settings for embeddings
  async_mode: threaded # or asyncio
  llm:
    api_key: sk-ySJkHe4。。。
    type: openai_embedding # or azure_openai_embedding
    model: text-embedding-3-small
    proxy: http://172.21.51.30:3128
    # api_base: https://<instance>.openai.azure.com
    # api_version: 2024-02-15-preview
    # organization: <organization_id>
    # deployment_name: <azure_model_deployment_name>
    # tokens_per_minute: 150_000 # set a leaky bucket throttle
    # requests_per_minute: 10_000 # set a leaky bucket throttle
    # max_retries: 10
    # max_retry_wait: 10.0
    # sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
    # concurrent_requests: 25 # the number of parallel inflight requests that may be made
# batch_size: 16 # the number of documents to send in a single request
    # batch_max_tokens: 8191 # the maximum number of tokens to send in a single request
    # target: required # or optional
chunks:
  size: 300
  overlap: 100
  group_by_columns: [id] # by default, we don't allow chunks to cross documents

input:
  type: file # or blob
  file_type: text # or csv
  base_dir: "input"
  file_encoding: utf-8
  file_pattern: ".*\\.txt$"

cache:
  type: file # or blob
  base_dir: "cache"
  # connection_string: <azure_blob_storage_connection_string>
  # container_name: <azure_blob_storage_container_name>

Logs and screenshots

graphrag.config.read_dotenv INFO Loading pipeline .env file
17:33:47,38 graphrag.index.cli INFO using default configuration: {
    "llm": {
        "api_key": "REDACTED, length 51",
        "type": "openai_chat",
        "model": "gpt-4-turbo-preview",
        "max_tokens": 4000,
        "request_timeout": 180.0,
        "api_base": null,
        "api_version": null,
        "proxy": null,
        "cognitive_services_endpoint": null,
        "deployment_name": null,
        "model_supports_json": true,
        "tokens_per_minute": 0,
        "requests_per_minute": 0,
        "max_retries": 10,
        "max_retry_wait": 10.0,
        "sleep_on_rate_limit_recommendation": true,
        "concurrent_requests": 25
    },
    "parallelization": {
        "stagger": 0.3,
        "num_threads": 50
    },
    "async_mode": "threaded",
    "root_dir": "./ragtest",
    "reporting": {
        "type": "file",
        "base_dir": "output/${timestamp}/reports",
        "storage_account_blob_url": null
    },
    "storage": {
        "type": "file",
        "base_dir": "output/${timestamp}/artifacts",
        "storage_account_blob_url": null
    },
    "cache": {
        "type": "file",
        "base_dir": "cache",
        "storage_account_blob_url": null
    },
    "input": {
        "type": "file",
        "file_type": "text",
        "base_dir": "input",
        "storage_account_blob_url": null,
        "encoding": "utf-8",
        "file_pattern": ".*\\.txt$",
        "file_filter": null,
        "source_column": null,
        "timestamp_column": null,
        "timestamp_format": null,
        "text_column": "text",
        "title_column": null,
        "document_attribute_columns": []
    },
    "embed_graph": {
        "enabled": false,
        "num_walks": 10,
        "walk_length": 40,
        "window_size": 2,
        "iterations": 3,
        "random_seed": 597832,
        "strategy": null
    },
    "embeddings": {
        "llm": {
            "api_key": "REDACTED, length 51",
            "type": "openai_embedding",
            "model": "text-embedding-3-small",
            "max_tokens": 4000,
            "request_timeout": 180.0,
            "api_base": null,
            "api_version": null,
            "proxy": "http://172.21.51.30:3128",
            "cognitive_services_endpoint": null,
            "deployment_name": null,
            "model_supports_json": null,
            "tokens_per_minute": 0,
            "requests_per_minute": 0,
            "max_retries": 10,
            "max_retry_wait": 10.0,
            "sleep_on_rate_limit_recommendation": true,
            "concurrent_requests": 25
        },
        "parallelization": {
            "stagger": 0.3,
            "num_threads": 50
        },
        "async_mode": "threaded",
        "batch_size": 16,
        "batch_max_tokens": 8191,
        "target": "required",
        "skip": [],
        "vector_store": null,
        "strategy": null
    },
    "chunks": {
        "size": 300,
        "overlap": 100,
        "group_by_columns": [
            "id"
        ],
        "strategy": null
        。。。。 
raise ValueError("Columns must be same length as key")
ValueError: Columns must be same length as key
18:26:56,914 graphrag.index.reporting.file_workflow_callbacks INFO Error running pipeline! details=None17:33:47,34

Additional Information

jgbradley1 commented 1 month ago

This is a known bug. The root cause is due to exceeding the TPM/RPM of the LLM too many times. Please set the tokens_per_minute and requests_per_minute config setting to an appropriate value other than 0. When set to 0, graphrag will run unthresholded. graphrag will make retry attempts, but eventually it will exceed the max retry limit. This causes a corruption in the underlying dataframe that is passed around inside graphrag.

We are looking at better error handling to improve this behavior.

zhangyanli-pro commented 1 month ago

This is a known bug. The root cause is due to exceeding the TPM/RPM of the LLM too many times. Please set the tokens_per_minute and requests_per_minute config setting to an appropriate value other than 0. When set to 0, graphrag will run unthresholded. graphrag will make retry attempts, but eventually it will exceed the max retry limit. This causes a corruption in the underlying dataframe that is passed around inside graphrag.

We are looking at better error handling to improve this behavior.

@jgbradley1 hi,Why is it that when I change the tokens_per_minute and requests_per_minute values in settings.yaml, but I see that the llm configuration item is not changed to the corresponding value in the log? Only the configuration item in embeddings llm can be modified successfully.

image image image image image
shadowwider commented 1 month ago

Can you use the proxy http://172.21.51.30:3128 normally? I checked the source code of the latest released version and it seems that the proxy parameter of openaiclient is not passed, and proxy cannot be used under LLM but api_proxy instead. I had to modify it to make it work.

ce-dataguy commented 1 month ago

I have a similar problem when running the indexing job

It continuously retries after hitting TMP threshold

The index job is stuck at 6.25% completion

Pipeline configuration : lm: type: azure_openai_chat api_base: $GRAPHRAG_API_BASE api_version: $GRAPHRAG_API_VERSION model: $GRAPHRAG_LLM_MODEL deployment_name: $GRAPHRAG_LLM_DEPLOYMENT_NAME cognitive_services_endpoint: $GRAPHRAG_COGNITIVE_SERVICES_ENDPOINT model_supports_json: True tokens_per_minute: 8000 requests_per_minute: 1 thread_count: 15 concurrent_requests: 1

parallelization: stagger: 0.75 num_threads: 2

async_mode: threaded

embeddings: async_mode: threaded llm: type: azure_openai_embedding api_base: $GRAPHRAG_API_BASE api_version: $GRAPHRAG_API_VERSION batch_size: 16 model: $GRAPHRAG_EMBEDDING_MODEL deployment_name: $GRAPHRAG_EMBEDDING_DEPLOYMENT_NAME cognitive_services_endpoint: $GRAPHRAG_COGNITIVE_SERVICES_ENDPOINT tokens_per_minute: 100000 concurrent_requests: 15 requests_per_minute: 1000 thread_count: 25 max_retries: 50 parallelization: stagger: 0.75 num_threads: 10 vector_store: type: azure_ai_search collection_name: PLACEHOLDER title_column: name overwrite: True url: $AI_SEARCH_URL audience: $AI_SEARCH_AUDIENCE

github-actions[bot] commented 3 weeks ago

This issue has been closed after being marked as stale for five days. Please reopen if needed.