dosubot[bot]
commented
1 month ago Hey @xiaohuozilalala! 👋 I'm here to help you with any bugs, questions, or contributions you have for the repository. Let me know how I can assist you while you wait for a human maintainer.
The JSON parsing error you're encountering is likely due to the presence of an 'items' key in the JSON output, which can break the parsing from open-source LLMs such as Zephyr. To resolve this issue, you need to ensure that the JSON output does not contain the 'items' key or handle it appropriately in your parsing logic.
Here's how you can modify the SubQuestionOutputParser class to handle this:
class SubQuestionOutputParser(BaseOutputParser):
def parse(self, output: str) -> Any:
json_dict = parse_json_markdown(output)
if not json_dict:
raise ValueError(f"No valid JSON found in output: {output}")
# Handle the 'items' key if present
if "items" in json_dict:
json_dict = json_dict["items"]
sub_questions = [SubQuestion.parse_obj(item) for item in json_dict]
return StructuredOutput(raw_output=output, parsed_output=sub_questions)
def format(self, prompt_template: str) -> str:
return prompt_templateThis modification ensures that if the 'items' key is present in the JSON output, it will be handled correctly, preventing the JSON parsing error.
Additionally, ensure that your JSON output is structured correctly and does not include any extraneous data that could cause parsing issues. If the problem persists, you may need to investigate why the LLM predictor is returning invalid JSON and address it at the source.
For a more robust solution, consider using the PydanticOutputParser class from the LlamaIndex framework, which provides a structured way to handle JSON parsing and formatting:
class PydanticOutputParser(ChainableOutputParser):
def __init__(self, output_cls: Type[Model], excluded_schema_keys_from_format: Optional[List] = None, pydantic_format_tmpl: str = PYDANTIC_FORMAT_TMPL) -> None:
self._output_cls = output_cls
self._excluded_schema_keys_from_format = excluded_schema_keys_from_format or []
self._pydantic_format_tmpl = pydantic_format_tmpl
@property
def output_cls(self) -> Type[Model]:
return self._output_cls
@property
def format_string(self) -> str:
return self.get_format_string(escape_json=True)
def get_format_string(self, escape_json: bool = True) -> str:
schema_dict = self._output_cls.schema()
for key in self._excluded_schema_keys_from_format:
del schema_dict[key]
schema_str = json.dumps(schema_dict)
output_str = self._pydantic_format_tmpl.format(schema=schema_str)
if escape_json:
return output_str.replace("{", "{{").replace("}", "}}")
else:
return output_str
def parse(self, text: str) -> Any:
json_str = extract_json_str(text)
return self._output_cls.parse_raw(json_str)
def format(self, query: str) -> str:
return query + "\n\n" + self.get_format_string(escape_json=True)This class ensures that the JSON output adheres to a specified schema, making it easier to parse and validate the output [1][2][3].
To continue talking to Dosu, mention @dosu.
xiaohuozilalala
Question Validation
Question
When I followed the official documentation guide "Llama3 Cookbook with Ollama", I encountered a problem in the example "Break Complex Questions down into Sub-Questions". When I tried to run the code according to the tutorial in the example, the compiler prompted an error in JSON parsing, which did not occur in the previous tutorials.
code show as below:
The compiler reports the following error:
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "E:\CodeProject\langchain1\pythonProject1\ollama-langchain-example.py", line 191, in
response = query_engine.query("Which albums did Drake release in his career?")
File "E:\CodeProject\conda_file\envs\langchain1\lib\site-packages\llama_index\core\instrumentation\dispatcher.py", line 198, in wrapper
result = func(*args, **kwargs)
File "E:\CodeProject\conda_file\envs\langchain1\lib\site-packages\llama_index\core\base\base_query_engine.py", line 51, in query
query_result = self._query(str_or_query_bundle)
File "E:\CodeProject\conda_file\envs\langchain1\lib\site-packages\llama_index\core\query_engine\sub_question_query_engine.py", line 145, in _query
sub_questions = self._question_gen.generate(self._metadatas, query_bundle)
File "E:\CodeProject\conda_file\envs\langchain1\lib\site-packages\llama_index\core\question_gen\llm_generators.py", line 81, in generate
parse = self._prompt.output_parser.parse(prediction)
File "E:\CodeProject\conda_file\envs\langchain1\lib\site-packages\llama_index\core\question_gen\output_parser.py", line 11, in parse
json_dict = parse_json_markdown(output)
File "E:\CodeProject\conda_file\envs\langchain1\lib\site-packages\llama_index\core\output_parsers\utils.py", line 54, in parse_json_markdown
raise OutputParserException(
llama_index.core.output_parsers.base.OutputParserException: Got invalid JSON object. Error: Extra data: line 21 column 1 (char 500) while scanning for the next token
found character '`' that cannot start any token
in "", line 21, column 1:
In the error message, there is an error message about JSON parsing. What could be the reason? . What also puzzles me is that the error message contains information about previously used tools (such as "uber_10k", "lyft_10k", etc.), which seem to be loaded, but I did not call them. Is there any solution? Thank you very much!