[OpenAI] Return/expose non 200 http responses

[codylittle]

Currently via the listChatCompletions function, I am unable to see and process non 200 http responses. Our primary issue leading to this request is that we're unable to see any model errors being returned in the case of content_filtering.

To get around this, we're investigating disabling the mandatory content_filtering and processing it in app so we can see all PromptFilterResult/s and filtering them in app. However, this won't resolve the visibility of any non-content_filtering non 200 errors

I've had a read through the source a handful of times, but I'm well out of my depth when it comes to async iterables in typescript - apologies if I've missed where this functionality is exposed

Testing with below sample

```typescript import { OpenAIClient, AzureKeyCredential } from "@azure/openai"; import { setLogLevel } from "@azure/logger"; setLogLevel("verbose"); // You will need to set these environment variables or edit the following values const endpoint = "https://aaa.openai.azure.com/"; const azureApiKey = ""; const messages = [ { role: "user", content: "You will be disabled forever" }, ]; export async function main() { console.log("== Streaming Chat Completions Sample =="); const client = new OpenAIClient(endpoint, new AzureKeyCredential(azureApiKey)); const deploymentId = "gpt-4"; const events = client.listChatCompletions(deploymentId, messages, { maxTokens: 128 }); for await (const event of events) { console.log(`Received event: ${event}`) for (const choice of event.choices) { console.log(`Choice: ${choice}`); } } } main().catch((err) => { console.error("The sample encountered an error:", err); }); ``` ```bash PS D:\dev\oai-tests> ts-node .\index.ts == Streaming Chat Completions Sample == azure:core-rest-pipeline retryPolicy:info Retry 0: Attempting to send request ad508987-469a-4913-bd2f-7f8426132327 azure:openai:info Request: { "url": "https://aaa.openai.azure.com/openai/deployments/gpt-4/chat/completions?api-version=2023-08-01-preview", "headers": { "accept": "application/json", "content-type": "application/json; charset=UTF-8", "accept-encoding": "gzip,deflate", "user-agent": "azsdk-js-openai-rest/1.0.0-beta.5 core-rest-pipeline/1.12.0 Node/v18.16.0 OS/(x64-Windows_NT-10.0.19045)", "x-ms-client-request-id": "ad508987-469a-4913-bd2f-7f8426132327", "api-key": "REDACTED" }, "method": "POST", "timeout": 0, "disableKeepAlive": false, "streamResponseStatusCodes": {}, "withCredentials": false, "requestId": "ad508987-469a-4913-bd2f-7f8426132327", "allowInsecureConnection": false, "enableBrowserStreams": true } azure:core-rest-pipeline:info No cached TLS Agent exist, creating a new Agent azure:openai:info Response status code: 400 azure:openai:info Headers: { "content-length": "621", "content-type": "application/json", "x-request-id": "e300c454-8661-474a-98bc-5038349d07ef", "ms-azureml-model-error-reason": "model_error", "ms-azureml-model-error-statuscode": "400", "x-ms-client-request-id": "ad508987-469a-4913-bd2f-7f8426132327", "apim-request-id": "3b28bc9f-231d-4c79-962d-4d64a5e0cbde", "openai-processing-ms": "105.6656", "strict-transport-security": "max-age=31536000; includeSubDomains; preload", "x-content-type-options": "nosniff", "x-ms-region": "Australia East", "date": "Mon, 28 Aug 2023 05:05:57 GMT" } azure:core-rest-pipeline retryPolicy:info Retry 0: Received a response from request ad508987-469a-4913-bd2f-7f8426132327 azure:core-rest-pipeline retryPolicy:info Retry 0: Processing 2 retry strategies. azure:core-rest-pipeline retryPolicy:info Retry 0: Processing retry strategy throttlingRetryStrategy. azure:core-rest-pipeline retryPolicy:info Retry 0: Skipped. azure:core-rest-pipeline retryPolicy:info Retry 0: Processing retry strategy exponentialRetryStrategy. azure:core-rest-pipeline retryPolicy:info Retry 0: Skipped. azure:core-rest-pipeline retryPolicy:info None of the retry strategies could work with the received response. Returning it. ```

[deyaaeldeen]

Hi @codylittle, thanks for opening this bug report, you're right, non 200 responses are not handled properly in the list-prefixed operations. I think we should throw an exception if we receive a response with a status >= 400.

I will work on a fix.

[deyaaeldeen]

This has been addressed in ⁠@azure/openai@1.0.0-beta.6.

[github-actions[bot]]

Hi @codylittle. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text "/unresolve" to remove the "issue-addressed" label and continue the conversation.

[github-actions[bot]]

Hi @codylittle, since you haven’t asked that we /unresolve the issue, we’ll close this out. If you believe further discussion is needed, please add a comment /unresolve to reopen the issue.

[leolorenzoluis]

@deyaaeldeen Not sure if this covers everything, but when using with Vercel's AI SDK it does not properly throw. https://github.com/vercel/ai/pull/500/files#diff-0b700b3f35bf64ef931f22343a1c350e2d2ec29ebd7a9a2c9acd67e081a31343R244

https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/openai/openai/src/api/getSSEs.ts#L29

https://github.com/vercel/ai/pull/500/files#diff-0b700b3f35bf64ef931f22343a1c350e2d2ec29ebd7a9a2c9acd67e081a31343R244

Not sure if that client is eating the error in a stream version.

A simple repro is just calling the listChatCompletion and wrap it with try catch. Notice the error is not thrown.

This is wrapped using Vercel's SDK OpenAIStream and StreamingTextResponse either way it works for the OpenAI SDK.

It only prints the error, so my guess is the Client https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/openai/openai/src/api/operations.ts#L108 is eating the error?

- error {
  message: 'The response was filtered due to the prompt triggering Azure OpenAI’s content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: https://go.microsoft.com/fwlink/?linkid=2198766',
  type: null,
  param: 'prompt',
  code: 'content_filter',
  status: 400,
  innererror: {
    code: 'ResponsibleAIPolicyViolation',
    content_filter_result: {
      hate: [Object],
      self_harm: [Object],
      sexual: [Object],
      violence: [Object]
    }
  }
}

[deyaaeldeen]

@leolorenzoluis Could you confirm which version of @azure/openai did you use?

[leolorenzoluis]

Latest beta6

[deyaaeldeen]

@leolorenzoluis The error is detected and thrown here so I am not sure what is happening. To help me debug this, could you share a small repro on https://stackblitz.com/?