Azure / semantic-kernel-bot-in-a-box

Extensible Semantic Kernel Bot Solution Accelerator
MIT License
56 stars 12 forks source link

getting error Chat history is too long to get completion. Try reducing the available functions #5

Open Naveen-Reddy237 opened 1 year ago

Naveen-Reddy237 commented 1 year ago

Hi @MarcoABCardoso I have imported two functions like below kernel.ImportFunctions(new KBSearchPlugin(_config, conversationData), "KBSearchPlugin"); kernel.ImportFunctions(new SearchPlugin(_config, conversationData), "SearchPlugin");

When I search any information from KBSearchPlugin it is generating observations in the response but at the end instead of final Answer it is giving error ChatHistory is too long to get a completion. Try reducing the available functions.

could you please advise, Thanks

Originally posted by @Naveen-Reddy237 in https://github.com/Azure/semantic-kernel-bot-in-a-box/issues/3#issuecomment-1794163574

MarcoABCardoso commented 1 year ago

Hello @Naveen-Reddy237!

Depending on the number and size of observations, the context may exceed your LLMs limits (4k for GPT 3.5 turbo, 8k for GPT-4). For example, if a large document is returned by your plugin.

You may get around this issue by either reducing the amount of data returned by your plugin, or using a model with larger context - for example, GPT-4-32k. You can also use the CONVERSATION_HISTORY_MAX_MESSAGES configuration to reduce the number of conversation turns kept in the assistant's memory.

Naveen-Reddy237 commented 1 year ago

Thanks for the response @MarcoABCardoso

I'm using gpt-4-32k model but still it is giving chat history too long error.

Could you please advise on how we can reduce the data returned by plugin?

and one more can we add this bot as plugin in chat copilot app? Thanks

MarcoABCardoso commented 1 year ago

Could you provide some more information about what your custom KBSearchPlugin is doing?

It's likely that the issue is that its functions are returning too many tokens at one time (for example, entire documents) which is then exceeding the token limit for a single Completions call.

This is a good place to learn more about document chunking for RAG pattern solutions: https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/azure-cognitive-search-outperforming-vector-search-with-hybrid/ba-p/3929167

Regarding integration with the Chat Copilot app, this may be done by leveraging the Direct Line channel from Azure Bot. IT exposes a REST API that may be used to initiate conversations, send messages and listen for conversation events. However, you'll be required to implement the logic to establish this communication from the Chat Copilot App.

More information here: https://learn.microsoft.com/en-us/azure/bot-service/rest-api/bot-framework-rest-direct-line-3-0-api-reference?view=azure-bot-service-4.0

Naveen-Reddy237 commented 1 year ago

Thanks for the response @MarcoABCardoso.

KBSearchPlugin is similar to SearchPlugin where we have created indexer that imports content from Azure Blob Storage which contains some pdf and word docs and makes it searchable in Azure Cognitive Search index. and I search any information from KBSearchPlugin it is generating observations in the response but at the end instead of final Answer it is giving error ChatHistory is too long to get a completion. Try reducing the available functions. Thanks