Regarding Third-party Custom Models for Azure API

[Ericc99]

I am having some problem when trying to use a thiry-party custom model with a customed base URL.

The following are my steps to where I am now, to give you a clue of what I was trying and what might be the problem.

1. Test my access

Before I start, I made sure my access is viable using the curl CLI. Everything is normal.

Essentailly, my testing command look like the following.

curl https://<THIRD_PARTY_BASE_URL>/openai/deployments/gpt-4o/completions?api-version=2024-06-01\
  -H "Content-Type: application/json" \
    -H "api-key: <MY_API_KEY>
    -d '{"messages": [{"role": "system", "content": "You are an Azure expert."}, 
                                  {"role": "user", "content": "What is OpenAI ?"}]}'

2. Try the custom model settings

So when I tried to use the "Custom Model" penal to set up my customed azur api, some problem surfaced.

If I choose Azure OpenAI as the provider, then there will be an error saying "Error Creating Model" if I have gpt-4o as model name and leaving the Base URL empty or not.

But if I choose the 3rd party (openai-format) as the provider than nothing comes back.

3. Look into developer console

When I look into the console, the POST request seems not to be shaped properly. They are formatted as an OpenAI request with a base url that is supposed to be Azure API.

# This is ths POST URL
https://<THIRD_PARTY_BASE_URL>/chat/completions/ 

Therefore, I tried to modify the Base URL so that the missing parts can be added in a Azure OpenAI style.

I modified the Base URL to be https://<THIRD_PARTY_BASE_URL>/openai/deployments/gpt-4o/ which makes my POST url https://<THIRD_PARTY_BASE_URL>/openai/deployments/gpt-4o/chat/completions/.

But there is no way I can get the api-version prompt onto that URL.

4. COR problem

Done with those twisted operations, I then encontered the CORS problem, saying:

# Error Message: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

So I turned CORS on and tried again.

This time I got no response and the console does not pop up any new errors.

Guess I'm down to the end of the road.

Conclusion

Best guess is that I have done something wrong.

Worst guess is 3rd-party Azure API is not yet supported.

Wild dream is that might as well just open up a curl CLI ish setting format so that we can plugin our curl-tested urls as the config.

Please help.

And thank you for your help in advance. :)

[Emt-lin]

@Ericc99 I think that When I add a custom model, these options will change based on the different provider.

[Emt-lin]

@Ericc99

Wild dream is that might as well just open up a curl CLI ish setting format so that we can plugin our curl-tested urls as the config.

config => curl-tested urls：easy

curl-tested urls => config： maybe hard

[Ericc99]

@Ericc99 I think that When I add a custom model, these options will change based on the different provider.

Yes you are correct.

Essentially my core question is that I am using a Customized Azure 3rd party API provided by my organization. So the base URL is different from that to the ordinary Azure API. Difference as follows.

# API I possess
https://<my_organization_name>.azure-api.net/openai/deployments/gpt-4o/completions?api-version=2024-06-01\

# Ordinary API 
https://<organization_name>.azure-api.com/openai/deployments/gpt-4o/completions?api-version=2024-06-01\

As you can see my organization have modified the base url to be .net instead of .com.

But as the default configuration, the plugin will still try to send POST reuest to the .com port.

Therefore I was forced to try the custom model to customize the Base URL.

But the problem is only when you select OpenAI as the provider will it allow you to input a 3rd party URL. As quote: "For 3rd party OpenAI Format endpoints only. Leave blank for other providers.".

So I'm stuck lol.

[Emt-lin]

@Ericc99 Can you provide the url and api key(only for test), I can't get the Azure API key.