getting error Chat history is too long to get completion. Try reducing the available functions

[Naveen-Reddy237]

Hi @MarcoABCardoso I have imported two functions like below kernel.ImportFunctions(new KBSearchPlugin(_config, conversationData), "KBSearchPlugin"); kernel.ImportFunctions(new SearchPlugin(_config, conversationData), "SearchPlugin");

When I search any information from KBSearchPlugin it is generating observations in the response but at the end instead of final Answer it is giving error ChatHistory is too long to get a completion. Try reducing the available functions.

could you please advise, Thanks

Originally posted by @Naveen-Reddy237 in https://github.com/Azure/semantic-kernel-bot-in-a-box/issues/3#issuecomment-1794163574

[MarcoABCardoso]

Hello @Naveen-Reddy237!

Depending on the number and size of observations, the context may exceed your LLMs limits (4k for GPT 3.5 turbo, 8k for GPT-4). For example, if a large document is returned by your plugin.

You may get around this issue by either reducing the amount of data returned by your plugin, or using a model with larger context - for example, GPT-4-32k. You can also use the CONVERSATION_HISTORY_MAX_MESSAGES configuration to reduce the number of conversation turns kept in the assistant's memory.

[Naveen-Reddy237]

Thanks for the response @MarcoABCardoso

I'm using gpt-4-32k model but still it is giving chat history too long error.

Could you please advise on how we can reduce the data returned by plugin?

and one more can we add this bot as plugin in chat copilot app? Thanks

[MarcoABCardoso]

Could you provide some more information about what your custom KBSearchPlugin is doing?

It's likely that the issue is that its functions are returning too many tokens at one time (for example, entire documents) which is then exceeding the token limit for a single Completions call.

This is a good place to learn more about document chunking for RAG pattern solutions: https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/azure-cognitive-search-outperforming-vector-search-with-hybrid/ba-p/3929167

Regarding integration with the Chat Copilot app, this may be done by leveraging the Direct Line channel from Azure Bot. IT exposes a REST API that may be used to initiate conversations, send messages and listen for conversation events. However, you'll be required to implement the logic to establish this communication from the Chat Copilot App.

More information here: https://learn.microsoft.com/en-us/azure/bot-service/rest-api/bot-framework-rest-direct-line-3-0-api-reference?view=azure-bot-service-4.0

[Naveen-Reddy237]

Thanks for the response @MarcoABCardoso.

KBSearchPlugin is similar to SearchPlugin where we have created indexer that imports content from Azure Blob Storage which contains some pdf and word docs and makes it searchable in Azure Cognitive Search index. and I search any information from KBSearchPlugin it is generating observations in the response but at the end instead of final Answer it is giving error ChatHistory is too long to get a completion. Try reducing the available functions. Thanks