Prompt and Request logging via APIM has an 8192 character limit

[rochaktarika123]

Describe the bug Using APIM for logging and audit purposes but there is an 8192 character limit which is usually hit because the request body contains the prompts, citations and collects the responses as well as the conversation builds. Once the 8192 character limit is hit, the request body json gets truncated and it is not structured and cannot be parsed. Is there a way to either stop the citations from logging ? (Short term solution) Any other way to avoid this issue?

To Reproduce Steps to reproduce the behavior:

1. Go to 'API Management Service and click on the API'
2. Click on 'Settings'
3. Scroll down to 'Diagnostic Logs -> Azure Monitor (check it) --> Additional Settings'
4. You can see the character limit

Expected behavior I am able to see the logs being captured but as soon as the limit hits , the json format breaks and I will not be able to record any conversation after that because the request body builds messages in an array

Screenshots

Configuration: Please provide the following

• Azure OpenAI model name and version (e.g. 'gpt-35-turbo-16k, version 0613')
• Is chat history enabled? Disabled
• Are you using data? If so, what data source? (e.g. Azure AI Search, Azure CosmosDB Mongo vCore, etc) Yes

Additional context Add any other context about the problem here.

[abhahn]

Hi @rochaktarika123 , just wanted to let you know that we currently have an item in our backlog for removing the citations and intents from the requests to our backend service. I'll mark it as a higher priority to get it fixed soon.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.