"modalities" parameter is automatically set and throws 400 Bad Request on Azure OpenAI

[sibbl]

Bug Report

Overview

Microsoft Azure's OpenAI API doesn't support the modalities property in the chat request.

The endpoint returns with a 400 Bad Request and the following body:

{
  "error": {
    "message": "Unrecognized request argument supplied: modalities",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}

To Reproduce

Send a very simple "hello world" textual ChatRequest to an Azure OpenAI deployment, leaving all optional parameters in their default values.

Expected behavior

I'd expect the optional modalities field not being auto-populated. It's currently automatically set to Text or to Text | Audio by default (depending on "audio" being present in the model name or not).

Additional context

Even the latest preview API version of Microsoft's OpenAI implementation doesn't include this parameter, so I don't expect it to go GA anytime soon - especially not on non-audio models.

[StephenHodgson]

prob not the only property that is gonna give your problems. I wish they would version their APIs better. Would make this a bit easier to deal with.