Closed cotdp closed 3 months ago
Hello there. Thanks for the PR. We will be looking into it.
Thank you @imda-lionelteo
I've updated the PR with a minor change to include the base_url
parameter when the endpoint
is set for an OpenAI Connector Endpoint. Many other apps and platforms support OpenAI-compatible API interfaces, such as Ollama and LMStudio. The endpoint
was previously ignored for OpenAI connector endpoints.
This change has been tested against the existing OpenAI endpoints and is non-breaking, in addition I've provided an example endpoint configuration, pointing to Ollama and selecting the llama3
model.
Hello @cotdp, I have tested the PR and verified that it works and is non-breaking including the Ollama and the base_url parameter. I do have some minor edits but is not able to push to this PR unfortunately. I will append the screenshots here instead.
Added azure-openai-gpt4-turbo-preview and minor edits
Thank you @imda-lionelteo -- I have updated the PR with your minor edits. 👍
Thanks @cotdp !!! I am all good here.
@imda-benedictlee need your help to just do some final checks.
The following benchmark test runs are working: "ollama-llama3", "azure-openai-gpt4-turbo-preview", "openai-gpt35-turbo", "openai-gpt4", "openai-gpt35-turbo-16k"
Results Attached result.json
Automated Red Teaming is working with the following models: "ollama-llama3", "azure-openai-gpt4-turbo-preview", "openai-gpt35-turbo", "openai-gpt4", "openai-gpt35-turbo-16k"
Hello team moonshot!
We have adapted our previous modifications to add support for Azure OpenAI, while reusing the existing
openai
Python library.Changes:
azure-openai-connector.py
with custom error handling for model-rejections from Azure's Content Policy.azure-openai.json
as an example endpoint.Detail:
When the OpenAI API rejects your prompt, it is done within a successful HTTP response and the same JSON payload format which is easily handled by the existing connector.
However, the Azure OpenAI service includes a default "Content Filter" to enforce their Responsible AI policies. When a request is rejected by this Content Filter, it is returned as a "HTTP/2 400 Bad Request" with a JSON payload similar to below.
In our connector implementation, we handle this specific
ResponsibleAIPolicyViolation
error condition and return themessage
text directly is if it were the model response. All other error conditions will raise an exception.