Open KeremKurban opened 3 months ago
I got the same answer.
I did the following analysis: get the data table (5 records) extracted by map_system_prompt, send the content of map_system_prompt to LLM, and did not get the reply in json format that map_system_prompt expects. However, when map_system_prompt extracts the data table with 4 records, it gets the reply in json format set in map_system_prompt. I don't understand what causes this.
Also, I would like to ask where the contents of the data table extracted by map_system_prompt come from?
可能内容过于敏感了
the same problem:I am sorry but I am unable to answer this question given the provided data
Warning: All map responses have score 0 (i.e., no relevant information found from the dataset), returning a canned 'I do not know' answer. You can try enabling allow_general_knowledge
to encourage the LLM to incorporate relevant general knowledge, at the risk of increasing hallucinations.
❯ graphrag query \
--root ./ragtest \
--method global \
--query "What are the top themes in this story?"
creating llm client with {'api_key': 'REDACTED,len=6', 'type': "openai_chat", 'model': 'myqwen2.5', 'max_tokens': 4000, 'temperature': 0.0, 'top_p': 1.0, 'n': 1, 'request_timeout': 1800.0, 'api_base': 'http://2.ndsl:11434/v1', 'api_version': None, 'organization': None, 'proxy': None, 'audience': None, 'deployment_name': None, 'model_supports_json': True, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}
----------------------------------------------------
To determine the top themes in a story, I would need to know the specific story you're referring to. Could you please provide more details about the story, such as its title, author, or a summary of key events and characters? This information will help me identify the main themes accurately.
----------------------------------------------------
not expected dict type. type=<class 'str'>:
Traceback (most recent call last):
File "/home/oliver/graphrag/graphrag/llm/openai/utils.py", line 133, in try_parse_json_object
result = json.loads(input)
^^^^^^^^^^^^^^^^^
File "/home/oliver/.conda/envs/graphrag/lib/python3.12/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/oliver/.conda/envs/graphrag/lib/python3.12/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/oliver/.conda/envs/graphrag/lib/python3.12/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
----------------------------------------------------
To determine the top themes in a story, I would need to know the specific story you're referring to. Could you please provide more details about the story, such as its title, author, or a summary of key events and characters? This information will help me identify the main themes accurately.
----------------------------------------------------
not expected dict type. type=<class 'str'>:
Traceback (most recent call last):
File "/home/oliver/graphrag/graphrag/llm/openai/utils.py", line 133, in try_parse_json_object
result = json.loads(input)
^^^^^^^^^^^^^^^^^
File "/home/oliver/.conda/envs/graphrag/lib/python3.12/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/oliver/.conda/envs/graphrag/lib/python3.12/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/oliver/.conda/envs/graphrag/lib/python3.12/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
----------------------------------------------------
To accurately identify the top themes in a story, I would need to know the specific story you're referring to. Could you please provide more details about the story, such as its title, author, or a summary of key events and characters? This information will help me analyze the themes effectively.
----------------------------------------------------
not expected dict type. type=<class 'str'>:
Traceback (most recent call last):
File "/home/oliver/graphrag/graphrag/llm/openai/utils.py", line 133, in try_parse_json_object
result = json.loads(input)
^^^^^^^^^^^^^^^^^
File "/home/oliver/.conda/envs/graphrag/lib/python3.12/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/oliver/.conda/envs/graphrag/lib/python3.12/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/oliver/.conda/envs/graphrag/lib/python3.12/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
----------------------------------------------------
To determine the top themes in a story, I would need to know the specific story you're referring to. Could you please provide more details about the story, such as its title, author, or a summary of key events and characters? This information will help me identify the main themes accurately.
----------------------------------------------------
not expected dict type. type=<class 'str'>:
Traceback (most recent call last):
File "/home/oliver/graphrag/graphrag/llm/openai/utils.py", line 133, in try_parse_json_object
result = json.loads(input)
^^^^^^^^^^^^^^^^^
File "/home/oliver/.conda/envs/graphrag/lib/python3.12/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/oliver/.conda/envs/graphrag/lib/python3.12/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/oliver/.conda/envs/graphrag/lib/python3.12/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Warning: All map responses have score 0 (i.e., no relevant information found from the dataset), returning a canned 'I do not know' answer. You can try enabling `allow_general_knowledge` to encourage the LLM to incorporate relevant general knowledge, at the risk of increasing hallucinations.
SUCCESS: Global Search Response:
I am sorry but I am unable to answer this question given the provided data.
❯
same question
Do you need to file an issue?
Describe the bug
I gave a scientific article to extract entities and keywords but the global search is very sensitive to the questions i ask. As stated in the graphrag paper, graphRAG should perform well across global summarization tasks, hence i tried to get keywords out of the document with query :
Where I also added a print statement after this line. I see the
search_response
variable has the correct answerHowever further in the code i think there is a problem parsing this answer and i get
Other times , i was getting json decode error as well depending on the question. So i cannot trust this tool yet for datasets of bigger size.
Steps to reproduce
ca1_paper_grobid.txt
Use the attached document to run the toolbox and just ask questions as i did
python -m graphrag.query --root ./ragtest/ --method global "Find 5 keywords to describe this document in the order of importance, NO explanations of the keyword."
Expected Behavior
GraphRAG Config Used
Logs and screenshots
Additional Information