Please describe the feature you'd like to see
The current error logging mechanism isn't efficient. Retry errors can be spammy and provide less insightful information on where the error occurred. Other times, client side errors also are propagated as server side errors (e.g. Status code 400 from Azure OpenAI API due to either content moderation filter being triggered where prompt is rejected, or prompt is too long exceeding certain # of tokens)
Example 1 for client side error catching:
BadRequestError('Error code: 400 - {\'error\': {\'message\': "This model\'s maximum
context length is 4096 tokens. However, your messages resulted in 4512 tokens.
Please reduce the length of the messages.", \'type\': \'invalid_request_error\',
\'param\': \'messages\', \'code\': \'context_length_exceeded\'}}')
Example 2 for client side error catching:
BadRequestError('Error code: 400 - {\'error\': {\'message\': "The response was filtered due to
the prompt triggering Azure OpenAI\'s content management policy. Please modify your prompt
and retry. To learn more about our content filtering policies please read our documentation: https://go.microsoft.com/fwlink/?linkid=2198766", \'type\': None, \'param\': \'prompt\',
\'code\': \'content_filter\', \'status\': 400}}')
Describe the solution you'd like
Improve error propagation and logging
Please describe the feature you'd like to see The current error logging mechanism isn't efficient. Retry errors can be spammy and provide less insightful information on where the error occurred. Other times, client side errors also are propagated as server side errors (e.g. Status code
400from Azure OpenAI API due to either content moderation filter being triggered where prompt is rejected, or prompt is too long exceeding certain # of tokens)Example 1 for client side error catching:
Example 2 for client side error catching:
Describe the solution you'd like Improve error propagation and logging
Related Issue
Acceptance Criteria