❌ use vllm create_final_community_reports error

[xxll88]

only when create_final_community_reports , display error almost when is finished 。

          ❌ create_final_community_reports

None ⠙ GraphRAG Indexer ├── Loading Input (InputFileType.text) - 1 files loaded (0 filtered) 100% ├── create_base_text_units ├── create_base_extracted_entities ├── create_summarized_entities ├── create_base_entity_graph ├── create_final_entities ├── create_final_nodes ├── create_final_communities ├── join_text_units_to_entity_ids ├── create_final_relationships ├── join_text_units_to_relationship_ids └── create_final_community_reports ❌ Errors occurred during the pipeline run, see logs for more details.

03:52:10,989 graphrag.index.reporting.file_workflow_callbacks INFO Error Invoking LLM details={'input': '\nYou are a helpful assistant responsible for > 01:07:09,532 graphrag.index.graph.extractors.community_reports.community_reports_extractor ERROR error generating community report

01:07:10,885 graphrag.index.reporting.file_workflow_callbacks INFO Community Report Extraction Error details=None 01:07:10,885 graphrag.index.verbs.graph.report.strategies.graph_intelligence.run_graph_intelligence WARNING No report found for community: 15 01:07:10,930 datashaper.workflow.workflow INFO executing verb window 01:07:10,931 datashaper.workflow.workflow ERROR Error executing verb "window" in create_final_community_reports: 'community'

vllm serve /mnt/vdb1/model/Qwen2-72B-Instruct-GPTQ-Int4 \ --host 0.0.0.0 \ --port 8000 \ --served-model-name qwen2-72b \ --max-model-len 12000 \ --gpu-memory-utilization 0.8 \ --dtype bfloat16 \ --enforce_eager

llm: api_key: "xx" type: openai_chat model: qwen2-72b api_base: http://10.x.x.x:8000/v1 model_supports_json: true max_tokens: 4000 request_timeout: 180.0

another question: only when create community_reports, GPU is at low load about 5% , query once only every 180s , so speed is very slow.

[xxll88]

11:17:11,575 graphrag.index.run ERROR error running workflow create_final_community_reports Traceback (most recent call last): File "C:\Users\shrnema\AppData\Local\Programs\Python\Python311\Lib\site-packages\graphrag\index\run.py", line 323, in run_pipeline result = await workflow.run(context, callbacks) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\shrnema\AppData\Local\Programs\Python\Python311\Lib\site-packages\datashaper\workflow\workflow.py", line 369, in run timing = await self._execute_verb(node, context, callbacks) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\shrnema\AppData\Local\Programs\Python\Python311\Lib\site-packages\datashaper\workflow\workflow.py", line 410, in _execute_verb result = node.verb.func(**verb_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\shrnema\AppData\Local\Programs\Python\Python311\Lib\site-packages\datashaper\engine\verbs\window.py", line 73, in window window = __window_function_mapwindow_operation

File "C:\Users\shrnema\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\core\frame.py", line 4102, in getitem
indexer = self.columns.get_loc(key)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\shrnema\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\core\indexes\range.py", line 417, in get_loc
raise KeyError(key)
KeyError: 'community'
11:17:11,578 graphrag.index.reporting.file_workflow_callbacks INFO Error running pipeline! details=None

[maslenkovas]

I was having similar issue, for me the solution was in changing the community_report.txt prompt. In my case the model was generating community reports in a broken json format because of double curly braces in the prompt. When I changed the prompt by putting single curly braces instead of double ones - the error got solved.

Here are my error logs): 11:09:56,781 graphrag.index.reporting.file_workflow_callbacks INFO Community Report Extraction Error details=None 11:09:56,781 graphrag.index.verbs.graph.report.strategies.graph_intelligence.run_graph_intelligence WARNING No report found for community: 0 11:10:23,321 httpx INFO HTTP Request: POST api_base/chat/completions "HTTP/1.1 200 OK" 11:10:23,322 graphrag.llm.openai.utils ERROR error loading json, json={{ "title": "Machine Learning and its Applications",

and here is the updated prompt part:

- TITLE: community's name that represents its key entities - title should be short but specific. When possible, include representative named entities in the title.
- SUMMARY: An executive summary of the community's overall structure, how its entities are related to each other, and significant information associated with its entities.
- IMPACT SEVERITY RATING: a float score between 0-10 that represents the severity of IMPACT posed by entities within the community.  IMPACT is the scored importance of a community.
- RATING EXPLANATION: Give a single sentence explanation of the IMPACT severity rating.
- DETAILED FINDINGS: A list of 5-10 key insights about the community. Each insight should have a short summary followed by multiple paragraphs of explanatory text grounded according to the grounding rules below. Be comprehensive.

Return output as a well-formed JSON-formatted string with the following format:
    {
        "title": <report_title>,
        "summary": <executive_summary>,
        "rating": <impact_severity_rating>,
        "rating_explanation": <rating_explanation>,
        "findings": [
            {
                "summary":<insight_1_summary>,
                "explanation": <insight_1_explanation>
            },
            {
                "summary":<insight_2_summary>,
                "explanation": <insight_2_explanation>
            }
        ]
    }```

[natoverse]

Consolidating alternate model issues here: https://github.com/microsoft/graphrag/issues/657

[keakon]

I switched to lmdeploy. It's about 2x faster then vllm when serving qwen2-7b and won't be stuck during create_final_community_reports.

[l4b4r4b4b4]

I was actually able to use functionary vllm server with a llama-3.1 AWQ quant and indexing as well as both local and global search works like a charm.