dosubot[bot]
commented
1 month ago Hello @arunnuve! I'm here to help you with the issue you're facing. If you need assistance, feel free to ask.
To resolve the JSON and Key co-routine errors when using the SubQuestionQueryEngine with the model microsoft/Phi-3-mini-4k-instruct in a Retrieval-Augmented Generation (RAG) setup with llama-index 0.10.57, you need to ensure that the asynchronous methods are correctly implemented and that the JSON payloads are properly handled.
Here are the key points to check and correct:
-
Ensure Asynchronous Methods are Correctly Implemented:
- Make sure that the
aquerymethod of the query engine is correctly awaited. - Ensure that the
asyncio.gatheris used properly to handle multiple asynchronous tasks.
- Make sure that the
-
Proper Handling of JSON Payloads:
- Ensure that the JSON payloads are correctly formatted and that the keys used in the payloads match the expected keys.
Here is a snippet of the relevant code with corrections:
async def _aquery(self, query_bundle: QueryBundle) -> RESPONSE_TYPE:
with self.callback_manager.event(
CBEventType.QUERY, payload={EventPayload.QUERY_STR: query_bundle.query_str}
) as query_event:
sub_questions = await self._question_gen.agenerate(
self._metadatas, query_bundle
)
colors = get_color_mapping([str(i) for i in range(len(sub_questions))])
if self._verbose:
print_text(f"Generated {len(sub_questions)} sub questions.\n")
tasks = [
self._aquery_subq(sub_q, color=colors[str(ind)])
for ind, sub_q in enumerate(sub_questions)
]
qa_pairs_all = await asyncio.gather(*tasks)
qa_pairs_all = cast(List[Optional[SubQuestionAnswerPair]], qa_pairs_all)
# filter out sub questions that failed
qa_pairs: List[SubQuestionAnswerPair] = list(filter(None, qa_pairs_all))
nodes = [self._construct_node(pair) for pair in qa_pairs]
source_nodes = [node for qa_pair in qa_pairs for node in qa_pair.sources]
response = await self._response_synthesizer.asynthesize(
query=query_bundle,
nodes=nodes,
additional_source_nodes=source_nodes,
)
query_event.on_end(payload={EventPayload.RESPONSE: response})
return response
async def _aquery_subq(
self, sub_q: SubQuestion, color: Optional[str] = None
) -> Optional[SubQuestionAnswerPair]:
try:
with self.callback_manager.event(
CBEventType.SUB_QUESTION,
payload={EventPayload.SUB_QUESTION: SubQuestionAnswerPair(sub_q=sub_q)},
) as event:
question = sub_q.sub_question
query_engine = self._query_engines[sub_q.tool_name]
if self._verbose:
print_text(f"[{sub_q.tool_name}] Q: {question}\n", color=color)
response = await query_engine.aquery(question)
response_text = str(response)
if self._verbose:
print_text(f"[{sub_q.tool_name}] A: {response_text}\n", color=color)
qa_pair = SubQuestionAnswerPair(
sub_q=sub_q, answer=response_text, sources=response.source_nodes
)
event.on_end(payload={EventPayload.SUB_QUESTION: qa_pair})
return qa_pair
except ValueError:
logger.warning(f"[{sub_q.tool_name}] Failed to run {question}")
return NoneEnsure that the query_engine.aquery method is correctly awaited and that the JSON payloads are correctly structured. This should help resolve the JSON and Key co-routine errors in your setup [1][2].
To continue talking to Dosu, mention @dosu.
arunnuve
Bug Description
I'm doing RAG using llama-index.The model is Phi3-mini-4k. I have experimented all the models that supports sub-queryengine. When comparing those models, I got pretty good results from Phi3-mini-4k. Even though there are issues like JSON error and Key co-routine error. For same input, I got these errors alternatively. Point to be note I have only ingested 20 documents to do this.
Version
llama-index == 0.10.57
Steps to Reproduce
You can take this as your example to generate the same errors: https://docs.llamaindex.ai/en/stable/examples/query_engine/sub_question_query_engine/ (Please use Open source LLM and LLM question generator for the purpose)
Relevant Logs/Tracbacks