search

TheAiSingularity / graphrag-local-ollama

Local models support for Microsoft's graphrag using ollama (llama3, mistral, gemma2 phi3)- LLM & Embedding extraction
MIT License
602 stars 88 forks source link

using llama3.1 as llm, no valid JSON? #45

Open babuqi opened 1 month ago

babuqi commented 1 month ago

{"type": "error", "data": "Community Report Extraction Error", "stack": "Traceback (most recent call last):\n File \"/home/zippo/GraphRAG/ollama/repo/graphrag-local-ollama/graphrag/index/graph/extractors/community_reports/community_reports_extractor.py\", line 58, in call\n await self._llm(\n File \"/home/zippo/GraphRAG/ollama/repo/graphrag-local-ollama/graphrag/llm/openai/json_parsing_llm.py\", line 34, in call\n result = await self._delegate(input, kwargs)\n File \"/home/zippo/GraphRAG/ollama/repo/graphrag-local-ollama/graphrag/llm/openai/openai_token_replacing_llm.py\", line 37, in call\n return await self._delegate(input, kwargs)\n File \"/home/zippo/GraphRAG/ollama/repo/graphrag-local-ollama/graphrag/llm/openai/openai_history_tracking_llm.py\", line 33, in call\n output = await self._delegate(input, kwargs)\n File \"/home/zippo/GraphRAG/ollama/repo/graphrag-local-ollama/graphrag/llm/base/caching_llm.py\", line 104, in call\n result = await self._delegate(input, kwargs)\n File \"/home/zippo/GraphRAG/ollama/repo/graphrag-local-ollama/graphrag/llm/base/rate_limiting_llm.py\", line 177, in call\n result, start = await execute_with_retry()\n File \"/home/zippo/GraphRAG/ollama/repo/graphrag-local-ollama/graphrag/llm/base/rate_limiting_llm.py\", line 159, in execute_with_retry\n async for attempt in retryer:\n File \"/home/zippo/anaconda3/envs/GraphRAG/lib/python3.10/site-packages/tenacity/asyncio/init.py\", line 166, in anext\n do = await self.iter(retry_state=self._retry_state)\n File \"/home/zippo/anaconda3/envs/GraphRAG/lib/python3.10/site-packages/tenacity/asyncio/init.py\", line 153, in iter\n result = await action(retry_state)\n File \"/home/zippo/anaconda3/envs/GraphRAG/lib/python3.10/site-packages/tenacity/_utils.py\", line 99, in inner\n return call(*args, kwargs)\n File \"/home/zippo/anaconda3/envs/GraphRAG/lib/python3.10/site-packages/tenacity/init.py\", line 398, in \n self._add_action_func(lambda rs: rs.outcome.result())\n File \"/home/zippo/anaconda3/envs/GraphRAG/lib/python3.10/concurrent/futures/_base.py\", line 451, in result\n return self.get_result()\n File \"/home/zippo/anaconda3/envs/GraphRAG/lib/python3.10/concurrent/futures/_base.py\", line 403, in get_result\n raise self._exception\n File \"/home/zippo/GraphRAG/ollama/repo/graphrag-local-ollama/graphrag/llm/base/rate_limiting_llm.py\", line 165, in execute_with_retry\n return await do_attempt(), start\n File \"/home/zippo/GraphRAG/ollama/repo/graphrag-local-ollama/graphrag/llm/base/rate_limiting_llm.py\", line 147, in do_attempt\n return await self._delegate(input, kwargs)\n File \"/home/zippo/GraphRAG/ollama/repo/graphrag-local-ollama/graphrag/llm/base/base_llm.py\", line 48, in call\n return await self._invoke_json(input, **kwargs)\n File \"/home/zippo/GraphRAG/ollama/repo/graphrag-local-ollama/graphrag/llm/openai/openai_chat_llm.py\", line 90, in _invoke_json\n raise RuntimeError(FAILED_TO_CREATE_JSON_ERROR)\nRuntimeError: Failed to generate valid JSON output\n", "source": "Failed to generate valid JSON output", "details": null}

sepmein commented 3 weeks ago

same error, I failed at the last step, create community

adimarco commented 3 weeks ago

Likewise. This is maybe because the 7B parameter llama3.1 model I'm running locally just doesn't cut it? I plan to spin up a GPU in the cloud and test with a larger model.

HRishabh95 commented 3 weeks ago

Same error , Anyone able to solve it?

jialanxin commented 1 week ago

I export the ollama modelile of llama3.1 and set the parameter “num_ctx” to 20480. Then the pipeline can work.

babuqi commented 1 week ago

I export the ollama modelile of llama3.1 and set the parameter “num_ctx” to 20480. Then the pipeline can work.

how can i find this parameter?

jialanxin commented 1 week ago

I export the ollama modelile of llama3.1 and set the parameter “num_ctx” to 20480. Then the pipeline can work.

how can i find this parameter?

see ollama‘s doc https://github.com/ollama/ollama/blob/main/docs/modelfile.md

babuqi commented 1 week ago

I export the ollama modelile of llama3.1 and set the parameter “num_ctx” to 20480. Then the pipeline can work.

how can i find this parameter?

see ollama‘s doc https://github.com/ollama/ollama/blob/main/docs/modelfile.md

thank you for you help! create a modelfile FROM llama3.1 PARAMETER num_ctx 20480

then create a new model from this modelfile,and use the model, the issue could be solved