dosubot[bot]
commented
1 week ago To resolve the issues with the SchemaLLMPathExtractor in TGI version 0.7.0, you need to address both the missing "await" in the achat_with_tools() function and the GenerationError related to the on-the-fly pydantic schema.
Issue 1: Missing "await" in achat_with_tools()
You need to ensure that the achat_with_tools() function is properly awaited. Here is the corrected code for the achat_with_tools() function:
async def apredict_and_call(
self,
tools: List["BaseTool"],
user_msg: Optional[Union[str, ChatMessage]] = None,
chat_history: Optional[List[ChatMessage]] = None,
verbose: bool = False,
allow_parallel_tool_calls: bool = False,
**kwargs: Any,
) -> "AgentChatResponse":
"""Predict and call the tool."""
from llama_index.core.tools.calling import acall_tool_with_selection
from llama_index.core.chat_engine.types import AgentChatResponse
if not self.metadata.is_function_calling_model:
return await super().apredict_and_call(
tools,
user_msg=user_msg,
chat_history=chat_history,
verbose=verbose,
**kwargs,
)
response = await self.achat_with_tools(
tools,
user_msg,
chat_history,
verbose=verbose,
allow_parallel_tool_calls=allow_parallel_tool_calls,
**kwargs,
)
tool_calls = self.get_tool_calls_from_response(response)
tool_tasks = [
acall_tool_with_selection(tool_call, tools, verbose=verbose)
for tool_call in tool_calls
]
tool_outputs = await asyncio.gather(*tool_tasks)
if allow_parallel_tool_calls:
output_text = "\n\n".join(
[tool_output.content for tool_output in tool_outputs]
)
return AgentChatResponse(response=output_text, sources=tool_outputs)
else:
if len(tool_outputs) > 1:
raise ValueError("Invalid")
return AgentChatResponse(
response=tool_outputs[0].content, sources=tool_outputs
)This code ensures that the achat_with_tools() function is properly awaited, which should resolve the missing "await" issue [1].
Issue 2: GenerationError Related to Pydantic Schema
The GenerationError is likely due to the on-the-fly pydantic schema generation in the SchemaLLMPathExtractor. Ensure that the schema is correctly defined and used. Here is an example of how the SchemaLLMPathExtractor handles schema validation and extraction:
class SchemaLLMPathExtractor(TransformComponent):
# ... (other parts of the class)
def _prune_invalid_triplets(self, kg_schema: Any) -> List[Triplet]:
"""Prune invalid triplets."""
assert isinstance(kg_schema, self.kg_schema_cls)
valid_triplets = []
for triplet in kg_schema.triplets:
subject = triplet.subject.name
subject_type = triplet.subject.type
relation = triplet.relation.type
obj = triplet.object.name
obj_type = triplet.object.type
# check relations
if relation not in self.kg_validation_schema.get(
subject_type, [relation]
) and relation not in self.kg_validation_schema.get(obj_type, [relation]):
continue
# remove self-references
if subject.lower() == obj.lower():
continue
subj_node = EntityNode(label=subject_type, name=subject)
obj_node = EntityNode(label=obj_type, name=obj)
rel_node = Relation(
label=relation, source_id=subj_node.id, target_id=obj_node.id
)
valid_triplets.append((subj_node, rel_node, obj_node))
return valid_triplets
async def _aextract(self, node: BaseNode) -> BaseNode:
"""Extract triplets from a node."""
assert hasattr(node, "text")
text = node.get_content(metadata_mode="llm")
try:
kg_schema = await self.llm.astructured_predict(
self.kg_schema_cls,
self.extract_prompt,
text=text,
max_triplets_per_chunk=self.max_triplets_per_chunk,
)
triplets = self._prune_invalid_triplets(kg_schema)
except ValueError:
triplets = []
existing_nodes = node.metadata.pop(KG_NODES_KEY, [])
existing_relations = node.metadata.pop(KG_RELATIONS_KEY, [])
metadata = node.metadata.copy()
for subj, rel, obj in triplets:
subj.properties = metadata
obj.properties = metadata
rel.properties = metadata
existing_relations.append(rel)
existing_nodes.append(subj)
existing_nodes.append(obj)
node.metadata[KG_NODES_KEY] = existing_nodes
node.metadata[KG_RELATIONS_KEY] = existing_relations
return node
async def acall(
self, nodes: List[BaseNode], show_progress: bool = False, **kwargs: Any
) -> List[BaseNode]:
"""Extract triplets from nodes async."""
jobs = []
for node in nodes:
jobs.append(self._aextract(node))
return await run_jobs(
jobs,
workers=self.num_workers,
show_progress=show_progress,
desc="Extracting paths from text with schema",
)This class ensures that the extracted paths conform to the specified schema and handles the extraction process efficiently using asynchronous methods [2].
By addressing these two issues, you should be able to resolve the problems with the SchemaLLMPathExtractor in TGI version 0.7.0.
To continue talking to Dosu, mention @dosu.
logan-markewich
mphipps2
Bug Description
``I'm using TGI (newest version 0.7.0) to host locally, and I'm trying to run the PropertyGraphIndex routines. SimpleLLMPathExtractor and ImplicitPathExtractor seem to be fine, but I'm having issues with SchemaLLMPathExtractor.
Issue 1) in llms/huggingface/TextGenerationInference.py on line 1044 in the achat_with_tools() function, the response is called like:
I believe there should be an "await" in front of that achat call. Without the "await" I end up with empty responses and the error shown below for issue 1. After adding the await I get past this point
Issue 2) after fixing the first issue, I end up with the issue 2 error shown below, which I think is related to the on the fly pydantic schema defined in the SchemaLLMPathExtractor? But maybe it's related to the TGI integration as well
Any idea what's going on here?
Version
0.10.45.post1 and for llama-index-llms-huggingface: 0.2.3
Steps to Reproduce
Relevant Logs/Tracbacks