intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, Phi, MiniCPM, etc.) on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, GraphRAG, DeepSpeed, vLLM, FastChat, Axolotl, etc.
Apache License 2.0
6.46k stars 1.24k forks source link

json.decoder.JSONDecodeError: GraphRAG with IPEX-LLM on Intel GPU #11626

Open shailesh837 opened 1 month ago

shailesh837 commented 1 month ago

hello, I am trying to replicate GraphRAG Demo on Intel Arc GPU 770, But getting below issue :

I am facing issue wit mistral :

12:33:38,271 httpx INFO HTTP Request: POST http://localhost:11434/v1/chat/completions "HTTP/1.1 200 OK"
12:33:38,272 graphrag.llm.base.rate_limiting_llm INFO perf - llm.chat "Process" with 6 retries took 16.932000000029802. input_tokens=2152, output_tokens=693
12:33:53,74 httpx INFO HTTP Request: POST http://localhost:11434/v1/chat/completions "HTTP/1.1 200 OK"
12:33:53,75 graphrag.llm.base.rate_limiting_llm INFO perf - llm.chat "Process" with 6 retries took 31.74200000008568. input_tokens=2059, output_tokens=547
12:35:07,794 httpx INFO HTTP Request: POST http://localhost:11434/v1/chat/completions "HTTP/1.1 200 OK"
12:35:07,796 graphrag.llm.base.rate_limiting_llm INFO perf - llm.chat "Process" with 6 retries took 106.37199999997392. input_tokens=2234, output_tokens=2984
12:35:07,804 datashaper.workflow.workflow INFO executing verb snapshot
12:35:07,808 datashaper.workflow.workflow INFO executing verb merge_graphs
12:35:07,813 datashaper.workflow.workflow INFO executing verb snapshot_rows
12:35:07,815 graphrag.index.emit.parquet_table_emitter INFO emitting parquet table create_base_extracted_entities.parquet
12:35:07,897 graphrag.index.run INFO Running workflow: create_summarized_entities...
12:35:07,897 graphrag.index.run INFO dependencies for create_summarized_entities: ['create_base_extracted_entities']
12:35:07,897 graphrag.index.run INFO read table from storage: create_base_extracted_entities.parquet
12:35:07,906 datashaper.workflow.workflow INFO executing verb summarize_descriptions
12:35:07,911 datashaper.workflow.workflow INFO executing verb snapshot_rows
12:35:07,912 graphrag.index.emit.parquet_table_emitter INFO emitting parquet table create_summarized_entities.parquet
12:35:07,994 graphrag.index.run INFO Running workflow: create_base_entity_graph...
12:35:07,994 graphrag.index.run INFO dependencies for create_base_entity_graph: ['create_summarized_entities']
12:35:07,994 graphrag.index.run INFO read table from storage: create_summarized_entities.parquet
12:35:08,5 datashaper.workflow.workflow INFO executing verb cluster_graph
12:35:08,5 graphrag.index.verbs.graph.clustering.cluster_graph WARNING Graph has no nodes
12:35:08,6 datashaper.workflow.workflow ERROR Error executing verb "cluster_graph" in create_base_entity_graph: Columns must be same length as key
Traceback (most recent call last):
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/datashaper/workflow/workflow.py", line 410, in _execute_verb
    result = node.verb.func(**verb_args)
  File "/home/spandey2/LLM_KG_RAG/graphrag-local-ollama/graphrag/index/verbs/graph/clustering/cluster_graph.py", line 102, in cluster_graph
    output_df[[level_to, to]] = pd.DataFrame(
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/pandas/core/frame.py", line 4299, in __setitem__
    self._setitem_array(key, value)
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/pandas/core/frame.py", line 4341, in _setitem_array
    check_key_length(self.columns, key, value)
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/pandas/core/indexers/utils.py", line 390, in check_key_length
    raise ValueError("Columns must be same length as key")
ValueError: Columns must be same length as key
12:35:08,7 graphrag.index.reporting.file_workflow_callbacks INFO Error executing verb "cluster_graph" in create_base_entity_graph: Columns must be same length as key details=None
12:35:08,7 graphrag.index.run ERROR error running workflow create_base_entity_graph
Traceback (most recent call last):
  File "/home/spandey2/LLM_KG_RAG/graphrag-local-ollama/graphrag/index/run.py", line 323, in run_pipeline
    result = await workflow.run(context, callbacks)
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/datashaper/workflow/workflow.py", line 369, in run
    timing = await self._execute_verb(node, context, callbacks)
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/datashaper/workflow/workflow.py", line 410, in _execute_verb
    result = node.verb.func(**verb_args)
  File "/home/spandey2/LLM_KG_RAG/graphrag-local-ollama/graphrag/index/verbs/graph/clustering/cluster_graph.py", line 102, in cluster_graph
    output_df[[level_to, to]] = pd.DataFrame(
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/pandas/core/frame.py", line 4299, in __setitem__
    self._setitem_array(key, value)
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/pandas/core/frame.py", line 4341, in _setitem_array
    check_key_length(self.columns, key, value)
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/pandas/core/indexers/utils.py", line 390, in check_key_length
    raise ValueError("Columns must be same length as key")
ValueError: Columns must be same length as key
12:35:08,7 graphrag.index.reporting.file_workflow_callbacks INFO Error running pipeline! details=None
๐Ÿš€ create_base_documents
                                 id                                         text_units                                        raw_content                      title
0  8af33f74cd8e0e4b0384f5bf5396d993  [7b4e128a12389cacb693c4d1cf7a7965, efd8fda36bf...  Introduction to Graph Neural Networks\nGraph N...              GNN_intro.txt
1  a15d1b96e67359498242ba415f8aa326  [e65eea82cd46a8251e3ecf779e46cb6e, ee0c1bc3dce...  Introduction to Transformer Neural Networks\nT...     Transformers_intro.txt
2  66ed8cbe18ccd47bbaef69aa492f2337  [72ee0a4be0a9109cffbb8d94f4253493, 4a25dab6bbc...  Introduction to Machine Learning\nMachine lear...  machinelearning_intro.txt
3  f5af7825fb7ca37fb6a81f68f4a9a45f  [e2083317ca3a8f0690bde0981dd98ea3, bc5189e2787...  Introduction to Convolutional Neural Networks\...              CNN_intro.txt
๐Ÿš€ create_final_documents
                                 id                                      text_unit_ids                                        raw_content                      title
0  8af33f74cd8e0e4b0384f5bf5396d993  [7b4e128a12389cacb693c4d1cf7a7965, efd8fda36bf...  Introduction to Graph Neural Networks\nGraph N...              GNN_intro.txt
1  a15d1b96e67359498242ba415f8aa326  [e65eea82cd46a8251e3ecf779e46cb6e, ee0c1bc3dce...  Introduction to Transformer Neural Networks\nT...     Transformers_intro.txt
2  66ed8cbe18ccd47bbaef69aa492f2337  [72ee0a4be0a9109cffbb8d94f4253493, 4a25dab6bbc...  Introduction to Machine Learning\nMachine lear...  machinelearning_intro.txt
3  f5af7825fb7ca37fb6a81f68f4a9a45f  [e2083317ca3a8f0690bde0981dd98ea3, bc5189e2787...  Introduction to Convolutional Neural Networks\...              CNN_intro.txt
โ ด GraphRAG Indexer
โ”œโ”€โ”€ Loading Input (text) - 4 files loaded (0 filtered) โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” 100% 0:00:00 0:00:00
โ”œโ”€โ”€ create_base_text_units
โ”œโ”€โ”€ create_base_extracted_entities
โ”œโ”€โ”€ create_summarized_entities
โ”œโ”€โ”€ create_base_entity_graph
โ”œโ”€โ”€ create_final_entities
โ”œโ”€โ”€ create_final_nodes
โ”œโ”€โ”€ create_final_communities
โ”œโ”€โ”€ join_text_units_to_entity_ids
โ”œโ”€โ”€ create_final_relationships
โ”œโ”€โ”€ join_text_units_to_relationship_ids
โ”œโ”€โ”€ create_final_community_reports
โ”œโ”€โ”€ create_final_text_units
โ”œโ”€โ”€ create_base_documents
โ””โ”€โ”€ create_final_documents
๐Ÿš€ All workflows completed successfully.
INFO: Reading settings from ragtest/settings.yaml
creating llm client with {'api_key': 'REDACTED,len=9', 'type': "openai_chat", 'model': 'llama2', 'max_tokens': 4000, 'temperature': 0.0, 'top_p': 1.0, 'request_timeout': 180.0, 'api_base': 'http://localhost:11434/v1', 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': True, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}
Error parsing search response json
Traceback (most recent call last):
  File "/home/spandey2/LLM_KG_RAG/graphrag-local-ollama/graphrag/query/structured_search/global_search/search.py", line 194, in _map_response_single_batch
    processed_response = self.parse_search_response(search_response)
  File "/home/spandey2/LLM_KG_RAG/graphrag-local-ollama/graphrag/query/structured_search/global_search/search.py", line 232, in parse_search_response
    parsed_elements = json.loads(search_response)["points"]
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

SUCCESS: Global Search Response: I am sorry but I am unable to answer this question given the provided data.
Oscilloscope98 commented 1 month ago

Hi @shailesh837,

Have you added the request_timeout configuration to ragtest/settings.yml as we mentioned in GraphRAG quickstart?

If not, would you mind adding this configuration, conduct the graph indexing based on the updated settings.yml, and try on the query again?

Please let us know for any further questions :)

aiChatGPT35User123 commented 1 month ago

hello, I am trying to replicate GraphRAG Demo on Intel Arc GPU 770, But getting below issue :

I am facing issue wit mistral :

12:33:38,271 httpx INFO HTTP Request: POST http://localhost:11434/v1/chat/completions "HTTP/1.1 200 OK"
12:33:38,272 graphrag.llm.base.rate_limiting_llm INFO perf - llm.chat "Process" with 6 retries took 16.932000000029802. input_tokens=2152, output_tokens=693
12:33:53,74 httpx INFO HTTP Request: POST http://localhost:11434/v1/chat/completions "HTTP/1.1 200 OK"
12:33:53,75 graphrag.llm.base.rate_limiting_llm INFO perf - llm.chat "Process" with 6 retries took 31.74200000008568. input_tokens=2059, output_tokens=547
12:35:07,794 httpx INFO HTTP Request: POST http://localhost:11434/v1/chat/completions "HTTP/1.1 200 OK"
12:35:07,796 graphrag.llm.base.rate_limiting_llm INFO perf - llm.chat "Process" with 6 retries took 106.37199999997392. input_tokens=2234, output_tokens=2984
12:35:07,804 datashaper.workflow.workflow INFO executing verb snapshot
12:35:07,808 datashaper.workflow.workflow INFO executing verb merge_graphs
12:35:07,813 datashaper.workflow.workflow INFO executing verb snapshot_rows
12:35:07,815 graphrag.index.emit.parquet_table_emitter INFO emitting parquet table create_base_extracted_entities.parquet
12:35:07,897 graphrag.index.run INFO Running workflow: create_summarized_entities...
12:35:07,897 graphrag.index.run INFO dependencies for create_summarized_entities: ['create_base_extracted_entities']
12:35:07,897 graphrag.index.run INFO read table from storage: create_base_extracted_entities.parquet
12:35:07,906 datashaper.workflow.workflow INFO executing verb summarize_descriptions
12:35:07,911 datashaper.workflow.workflow INFO executing verb snapshot_rows
12:35:07,912 graphrag.index.emit.parquet_table_emitter INFO emitting parquet table create_summarized_entities.parquet
12:35:07,994 graphrag.index.run INFO Running workflow: create_base_entity_graph...
12:35:07,994 graphrag.index.run INFO dependencies for create_base_entity_graph: ['create_summarized_entities']
12:35:07,994 graphrag.index.run INFO read table from storage: create_summarized_entities.parquet
12:35:08,5 datashaper.workflow.workflow INFO executing verb cluster_graph
12:35:08,5 graphrag.index.verbs.graph.clustering.cluster_graph WARNING Graph has no nodes
12:35:08,6 datashaper.workflow.workflow ERROR Error executing verb "cluster_graph" in create_base_entity_graph: Columns must be same length as key
Traceback (most recent call last):
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/datashaper/workflow/workflow.py", line 410, in _execute_verb
    result = node.verb.func(**verb_args)
  File "/home/spandey2/LLM_KG_RAG/graphrag-local-ollama/graphrag/index/verbs/graph/clustering/cluster_graph.py", line 102, in cluster_graph
    output_df[[level_to, to]] = pd.DataFrame(
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/pandas/core/frame.py", line 4299, in __setitem__
    self._setitem_array(key, value)
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/pandas/core/frame.py", line 4341, in _setitem_array
    check_key_length(self.columns, key, value)
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/pandas/core/indexers/utils.py", line 390, in check_key_length
    raise ValueError("Columns must be same length as key")
ValueError: Columns must be same length as key
12:35:08,7 graphrag.index.reporting.file_workflow_callbacks INFO Error executing verb "cluster_graph" in create_base_entity_graph: Columns must be same length as key details=None
12:35:08,7 graphrag.index.run ERROR error running workflow create_base_entity_graph
Traceback (most recent call last):
  File "/home/spandey2/LLM_KG_RAG/graphrag-local-ollama/graphrag/index/run.py", line 323, in run_pipeline
    result = await workflow.run(context, callbacks)
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/datashaper/workflow/workflow.py", line 369, in run
    timing = await self._execute_verb(node, context, callbacks)
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/datashaper/workflow/workflow.py", line 410, in _execute_verb
    result = node.verb.func(**verb_args)
  File "/home/spandey2/LLM_KG_RAG/graphrag-local-ollama/graphrag/index/verbs/graph/clustering/cluster_graph.py", line 102, in cluster_graph
    output_df[[level_to, to]] = pd.DataFrame(
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/pandas/core/frame.py", line 4299, in __setitem__
    self._setitem_array(key, value)
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/pandas/core/frame.py", line 4341, in _setitem_array
    check_key_length(self.columns, key, value)
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/site-packages/pandas/core/indexers/utils.py", line 390, in check_key_length
    raise ValueError("Columns must be same length as key")
ValueError: Columns must be same length as key
12:35:08,7 graphrag.index.reporting.file_workflow_callbacks INFO Error running pipeline! details=None
๐Ÿš€ create_base_documents
                                 id                                         text_units                                        raw_content                      title
0  8af33f74cd8e0e4b0384f5bf5396d993  [7b4e128a12389cacb693c4d1cf7a7965, efd8fda36bf...  Introduction to Graph Neural Networks\nGraph N...              GNN_intro.txt
1  a15d1b96e67359498242ba415f8aa326  [e65eea82cd46a8251e3ecf779e46cb6e, ee0c1bc3dce...  Introduction to Transformer Neural Networks\nT...     Transformers_intro.txt
2  66ed8cbe18ccd47bbaef69aa492f2337  [72ee0a4be0a9109cffbb8d94f4253493, 4a25dab6bbc...  Introduction to Machine Learning\nMachine lear...  machinelearning_intro.txt
3  f5af7825fb7ca37fb6a81f68f4a9a45f  [e2083317ca3a8f0690bde0981dd98ea3, bc5189e2787...  Introduction to Convolutional Neural Networks\...              CNN_intro.txt
๐Ÿš€ create_final_documents
                                 id                                      text_unit_ids                                        raw_content                      title
0  8af33f74cd8e0e4b0384f5bf5396d993  [7b4e128a12389cacb693c4d1cf7a7965, efd8fda36bf...  Introduction to Graph Neural Networks\nGraph N...              GNN_intro.txt
1  a15d1b96e67359498242ba415f8aa326  [e65eea82cd46a8251e3ecf779e46cb6e, ee0c1bc3dce...  Introduction to Transformer Neural Networks\nT...     Transformers_intro.txt
2  66ed8cbe18ccd47bbaef69aa492f2337  [72ee0a4be0a9109cffbb8d94f4253493, 4a25dab6bbc...  Introduction to Machine Learning\nMachine lear...  machinelearning_intro.txt
3  f5af7825fb7ca37fb6a81f68f4a9a45f  [e2083317ca3a8f0690bde0981dd98ea3, bc5189e2787...  Introduction to Convolutional Neural Networks\...              CNN_intro.txt
โ ด GraphRAG Indexer
โ”œโ”€โ”€ Loading Input (text) - 4 files loaded (0 filtered) โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” 100% 0:00:00 0:00:00
โ”œโ”€โ”€ create_base_text_units
โ”œโ”€โ”€ create_base_extracted_entities
โ”œโ”€โ”€ create_summarized_entities
โ”œโ”€โ”€ create_base_entity_graph
โ”œโ”€โ”€ create_final_entities
โ”œโ”€โ”€ create_final_nodes
โ”œโ”€โ”€ create_final_communities
โ”œโ”€โ”€ join_text_units_to_entity_ids
โ”œโ”€โ”€ create_final_relationships
โ”œโ”€โ”€ join_text_units_to_relationship_ids
โ”œโ”€โ”€ create_final_community_reports
โ”œโ”€โ”€ create_final_text_units
โ”œโ”€โ”€ create_base_documents
โ””โ”€โ”€ create_final_documents
๐Ÿš€ All workflows completed successfully.
INFO: Reading settings from ragtest/settings.yaml
creating llm client with {'api_key': 'REDACTED,len=9', 'type': "openai_chat", 'model': 'llama2', 'max_tokens': 4000, 'temperature': 0.0, 'top_p': 1.0, 'request_timeout': 180.0, 'api_base': 'http://localhost:11434/v1', 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': True, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}
Error parsing search response json
Traceback (most recent call last):
  File "/home/spandey2/LLM_KG_RAG/graphrag-local-ollama/graphrag/query/structured_search/global_search/search.py", line 194, in _map_response_single_batch
    processed_response = self.parse_search_response(search_response)
  File "/home/spandey2/LLM_KG_RAG/graphrag-local-ollama/graphrag/query/structured_search/global_search/search.py", line 232, in parse_search_response
    parsed_elements = json.loads(search_response)["points"]
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/spandey2/miniconda3/envs/graphrag-local-ollama/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

SUCCESS: Global Search Response: I am sorry but I am unable to answer this question given the provided data.

ไฝ ๅฅฝ๏ผŒ่ฟ™ไธช้—ฎ้ข˜ไฝ ่งฃๅ†ณไบ†ๅ—