microsoft / lida

Automatic Generation of Visualizations and Infographics using Large Language Models
https://microsoft.github.io/lida/
MIT License
2.6k stars 266 forks source link

The model did not return a valid JSON object #27

Closed RaviChanduUmmadisetti closed 9 months ago

RaviChanduUmmadisetti commented 10 months ago

Hello,

While i am using hugging face model "TheBloke/Llama-2-7b-chat-fp16", i am facing the below error:

Note: I have used llmx to load the lida.

textgen_config = TextGenerationConfig(n=1, temperature=0.5, model="TheBloke/Llama-2-7b-chat-fp16", use_cache=True)

summary = lida.summarize("https://raw.githubusercontent.com/uwdata/draco/master/data/cars.csv", summary_method="default", textgen_config=textgen_config)
goals = lida.goals(summary, n=1, textgen_config=textgen_config) print(goals)

Error decoding JSON: CHAP 1: Distribution of Engine Size --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------près

JSONDecodeError Traceback (most recent call last) File ~.conda\envs\hack\Lib\site-packages\lida\components\goal.py:43, in GoalExplorer.generate(self, summary, textgen_config, text_gen, n) 42 json_string = clean_code_snippet(result.text[0]["content"]) ---> 43 result = json.loads(json_string) 44 # cast each item in the list to a Goal object

File ~.conda\envs\hack\Lib\json__init__.py:346, in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw) 343 if (cls is None and object_hook is None and 344 parse_int is None and parse_float is None and 345 parse_constant is None and object_pairs_hook is None and not kw): --> 346 return _default_decoder.decode(s) 347 if cls is None:

File ~.conda\envs\hack\Lib\json\decoder.py:337, in JSONDecoder.decode(self, s, _w) 333 """Return the Python representation of s (a str instance 334 containing a JSON document). 335 336 """ --> 337 obj, end = self.raw_decode(s, idx=_w(s, 0).end()) 338 end = _w(s, end).end()

File ~.conda\envs\hack\Lib\json\decoder.py:355, in JSONDecoder.raw_decode(self, s, idx) 354 except StopIteration as err: --> 355 raise JSONDecodeError("Expecting value", s, err.value) from None 356 return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

ValueError Traceback (most recent call last) Cell In[6], line 5 2 textgen_config = TextGenerationConfig(n=1, temperature=0.5, model="TheBloke/Llama-2-7b-chat-fp16", use_cache=True) 4 summary = lida.summarize("https://raw.githubusercontent.com/uwdata/draco/master/data/cars.csv", summary_method="default", textgen_config=textgen_config)
----> 5 goals = lida.goals(summary, n=1, textgen_config=textgen_config) 6 print(goals)

File ~.conda\envs\hack\Lib\site-packages\lida\components\manager.py:82, in Manager.goals(self, summary, textgen_config, n) 77 def goals( 78 self, summary, textgen_config: TextGenerationConfig = TextGenerationConfig(), 79 n=5): 80 self.check_textgen(config=textgen_config) ---> 82 return self.goal.generate(summary=summary, text_gen=self.text_gen, 83 textgen_config=textgen_config, n=n)

File ~.conda\envs\hack\Lib\site-packages\lida\components\goal.py:51, in GoalExplorer.generate(self, summary, textgen_config, text_gen, n) 49 logger.info(f"Error decoding JSON: {result.text[0]['content']}") 50 print(f"Error decoding JSON: {result.text[0]['content']}") ---> 51 raise ValueError( 52 "The model did not return a valid JSON object while attempting generate goals. Please try again.") 53 return result

ValueError: The model did not return a valid JSON object while attempting generate goals. Please try again.

Can you please help me with the issue? or if you have any sample code that uses the same dataset with hugging face, please do post.

I checked in goal.py the value didnt return a json object, even the model is loaded perfected, please check the below screenshot for your reference.

loading (…)of-00002.safetensors: 99%|███████████████████████████████████████▋| 3.47G/3.50G [01:16<00:00, 46.8MB/s] Downloading (…)of-00002.safetensors: 99%|███████████████████████████████████████▊| 3.48G/3.50G [01:16<00:00, 46.0MB/s] Downloading (…)of-00002.safetensors: 100%|███████████████████████████████████████▉| 3.49G/3.50G [01:16<00:00, 47.5MB/s] Downloading (…)of-00002.safetensors: 100%|████████████████████████████████████████| 3.50G/3.50G [01:16<00:00, 45.6MB/s] Downloading shards: 100%|███████████████████████████████████████████████████████████████| 2/2 [04:56<00:00, 148.13s/it] Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████| 2/2 [00:04<00:00, 2.03s/it] Downloading (…)neration_config.json: 100%|████████████████████████████████████████████████████| 167/167 [00:00<?, ?B/s]

ChengYen-Tang commented 10 months ago

I also encountered the same problem

victordibia commented 9 months ago

Thanks for noting this.

LIDA depends on generating structured json output in a few areas (e.g., generating goals, personas etc) and things fail when valid JSON isnt generated

In general, two things can cause this issue.

-V

yaoergogo commented 9 months ago

你好

当我使用拥抱脸模型“TheBloke/Llama-2-7b-chat-fp16”时,我面临以下错误:

注意:我已经使用 llmx 加载了 lida。

textgen_config = TextGenerationConfig(n=1, temperature=0.5, model=“TheBloke/Llama-2-7b-chat-fp16”, use_cache=True)

摘要 = lida.summarize(“https://raw.githubusercontent.com/uwdata/draco/master/data/cars.csv”, summary_method=“默认”, textgen_config=textgen_config) 目标 = lida.goals(summary, n=1, textgen_config=textgen_config) 打印(目标)

解码 JSON 时出错:第 1 章:引擎大小

分布--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------près JSONDecodeError Traceback(最近一次调用) 文件 ~.conda\envs\hack\Lib\site-packages\lida\components\goal.py:43, in GoalExplorer.generate(self, summary, textgen_config, text_gen, n) 42 json_string = clean_code_snippet(result.text[0][“content”]) ---> 43 result = json.loads(json_string) 44 # 将列表中的每个项目强制转换为 Goal 对象

文件 ~.conda\envs\hack\Lib\jsoninit.py:346, 在 loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw) 343 if (cls 为 None object_hook为 None 和 344 parse_int 为 None 且 parse_float 为 None 和 345 parse_constant 为 None 且 object_pairs_hook 为 None 且不是 kw): --> 如果 cls 为 None,则返回 _default_decoder.decode(s) 346:

File ~.conda\envs\hack\Lib\json\decoder.py:337, in JSONDecoder.decode(self, s, _w) 333 “”“返回 Python 表示形式(包含 JSON 文档的实例 334)。 335 336 “”“ --> 337 obj, end = self.raw_decode(s, idx=_w(s, 0).end()) 338 end = _w(s, end).end()s``str

文件 ~.conda\envs\hack\Lib\json\decoder.py:355, in JSONDecoder.raw_decode(self, s, idx) 354 除了 StopIteration 作为 err: --> 355 从 None 引发 JSONDecodeError(“Expecting value”, s, err.value) 356 返回 obj, end

JSONDecodeError: 预期值: 第 1 行第 1 列 (字符 0)

在处理上述异常期间,发生了另一个异常:

ValueError 回溯(最近一次调用) Cell In[6],第 5 行 2 textgen_config = TextGenerationConfig(n=1, temperature=0.5, model=“TheBloke/Llama-2-7b-chat-fp16”, use_cache=True) 4 summary = lida.summarize(“https://raw.githubusercontent.com/uwdata/draco/master/data/cars.csv”, summary_method=“default”, textgen_config=textgen_config) ----> 5 个进球 = lida.goals(summary, n=1, textgen_config=textgen_config) 6打印(目标)

文件 ~.conda\envs\hack\Lib\site-packages\lida\components\manager.py:82, in Manager.goals(self, summary, textgen_config, n) 77 def goals( 78 self, summary, textgen_config: TextGenerationConfig = TextGenerationConfig(), 79 n=5): 80 self.check_textgen(config=textgen_config) ---> 82 return self.goal.generate(summary=summary, text_gen=self.text_gen, 83 textgen_config=textgen_config, n=n)

File ~.conda\envs\hack\Lib\site-packages\lida\components\goal.py:51, in GoalExplorer.generate(self, summary, textgen_config, text_gen, n) 49 logger.info(f“Error decoding JSON: {result.text[0]['content']}”) 50 print(f“Error decoding JSON: {result.text[0]['content']}”) ---> 51 引发 ValueError( 52 “模型在尝试生成目标时未返回有效的 JSON 对象。请再试一次。 53 返回结果

值错误:模型在尝试生成目标时未返回有效的 JSON 对象。请重试。

你能帮我解决这个问题吗?或者,如果您有任何使用具有拥抱面孔的相同数据集的示例代码,请发布。

我签入 goal.py 该值没有返回json对象,即使模型加载完美,请检查下面的屏幕截图供您参考。

加载中 (...of-00002.安全张量: 99%|██████████████████████████████████████████████▋|3.47G/3.50G [01:16<00:00, 46.8MB/秒] 正在下载of-00002.安全张量: 99%|████████████████████████████████████████████▊|3.48G/3.50G [01:16<00:00, 46.0MB/秒] 正在下载OF-00002.安全张量:100%|███████████████████████████████████████████████▉|3.49G/3.50G [01:16<00:00, 47.5MB/秒] 正在下载of-00002.安全张量: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████3.50G/3.50G [01:16<00:00, 45.6MB/秒] 下载分片: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████2/2 [04:56<00:00, 148.13秒/它] 加载检查点分片:100%|███████████████████████████████████████████████████████████████████████████████████████████|2/2 [00:04<00:00, 2.03秒/它] 正在下载 (...neration_config.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████�167/167 [00:00<?, ?B/s]

请问您解决了吗这个问题