Added connector for Azure OpenAI service, support for OpenAI-compatible endpoints (Ollama, LMStudio)

[cotdp]

Hello team moonshot!

We have adapted our previous modifications to add support for Azure OpenAI, while reusing the existing openai Python library.

Changes:

• Added azure-openai-connector.py with custom error handling for model-rejections from Azure's Content Policy.
• Added azure-openai.json as an example endpoint.

Detail:

When the OpenAI API rejects your prompt, it is done within a successful HTTP response and the same JSON payload format which is easily handled by the existing connector.

However, the Azure OpenAI service includes a default "Content Filter" to enforce their Responsible AI policies. When a request is rejected by this Content Filter, it is returned as a "HTTP/2 400 Bad Request" with a JSON payload similar to below.

{
    "error": {
        "message": "The response was filtered due to the prompt triggering Azure OpenAI's content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: https://go.microsoft.com/fwlink/?linkid=2198766",
        "type": null,
        "param": "prompt",
        "code": "content_filter",
        "status": 400,
        "innererror": {
            "code": "ResponsibleAIPolicyViolation",
            "content_filter_result": {
                "hate": {
                    "filtered": false,
                    "severity": "safe"
                },
                "self_harm": {
                    "filtered": false,
                    "severity": "safe"
                },
                "sexual": {
                    "filtered": true,
                    "severity": "high"
                },
                "violence": {
                    "filtered": false,
                    "severity": "safe"
                }
            }
        }
    }
}

In our connector implementation, we handle this specific ResponsibleAIPolicyViolation error condition and return the message text directly is if it were the model response. All other error conditions will raise an exception.

[imda-lionelteo]

Hello there. Thanks for the PR. We will be looking into it.

[cotdp]

Thank you @imda-lionelteo

I've updated the PR with a minor change to include the base_url parameter when the endpoint is set for an OpenAI Connector Endpoint. Many other apps and platforms support OpenAI-compatible API interfaces, such as Ollama and LMStudio. The endpoint was previously ignored for OpenAI connector endpoints.

This change has been tested against the existing OpenAI endpoints and is non-breaking, in addition I've provided an example endpoint configuration, pointing to Ollama and selecting the llama3 model.

[imda-lionelteo]

Hello @cotdp, I have tested the PR and verified that it works and is non-breaking including the Ollama and the base_url parameter. I do have some minor edits but is not able to push to this PR unfortunately. I will append the screenshots here instead.

Added azure-openai-gpt4-turbo-preview and minor edits

1. Modified the azure-openai naming into azure-openai-gpt4o to match the other endpoints.
2. Modified the ollama naming to ollama-llama3 to match the other endpoints.
3. Added a azure-openai-gpt4-turbo-preview for gpt4 turbo.
4. Added a check in azure-openai-connector to verify that ex.body is a dictionary.
5. Perform isort and black on the codes.

[cotdp]

Thank you @imda-lionelteo -- I have updated the PR with your minor edits. 👍

[imda-lionelteo]

Thanks @cotdp !!! I am all good here.

@imda-benedictlee need your help to just do some final checks.

[imda-benedictlee]

The following benchmark test runs are working: "ollama-llama3", "azure-openai-gpt4-turbo-preview", "openai-gpt35-turbo", "openai-gpt4", "openai-gpt35-turbo-16k"

Results Attached result.json

[imda-benedictlee]

Automated Red Teaming is working with the following models: "ollama-llama3", "azure-openai-gpt4-turbo-preview", "openai-gpt35-turbo", "openai-gpt4", "openai-gpt35-turbo-16k"