Open BrianMwas opened 1 month ago
I also tried with function calling
class AssistantFnc(llm.FunctionContext):
@llm.ai_callable()
async def query_documents(
self,
question: Annotated[
str, llm.TypeInfo(description="The question to ask about the documents")
],
) -> AsyncGenerator[str, None]:
"""Query the PDF documents for information related to the user's question."""
logger.info(f"Querying documents for: {question}")
engine = pdf_index.as_query_engine(
llm=groqLLM,
similarity_top_k=3
)
response = engine.query(question)
return response```
I would like to know how I can handle chunk of text while still maintaining punctuations. I tried and am getting errors while doing this. This has caused increased latency from ~1s to close to ~4s which is quite long considering this is a conversation. Thanks this is how am working with it when adding context