How to use with Azure?

[AlaShibanAtKlo]

Great work on the CLI! How do I configure it to use our azure openai deployment? Thanks!

[kardolus]

Hey @AlaShibanAtKlo! Super stoked you reached out. 😊 I've been crossing my fingers for someone with Azure access to pop up since I don't have direct access to Azure myself. I think this CLI should play nice with Azure. Mind giving these settings a try?

Put this into your ~/.chatgpt-cli/config.yaml:

name: azure
api_key: <your api key>
model: gpt-4
max_tokens: 2048
temperature: 1
top_p: 1
frequency_penalty: 0
presence_penalty: 0
url: https://<your-resource-name>.api.azure-openai.net
completions_path: /v1/models/gpt-4:generate
auth_header: Authorization
auth_token_prefix: 'Bearer '

Heads up, the name here (azure) is the prefix for any env vars you want to slot in (like AZURE_API_KEY for api_key).

If things go sideways, it would be amazing if you could provide the output of this cURL:

curl -X POST \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <your api key>" \
-d '{"model": "gpt-4", "prompt": "Hello, how are you?", "max_tokens": 2048}' \
https://<your-resource-name>.api.azure-openai.net/v1/models/gpt-4:generate

I could use the output to tweak the CLI for Azure compatibility. Fingers crossed it just works, but if not, let's crack this together!

[kardolus]

@AlaShibanAtKlo bumping this in case you missed it.

[AlaShibanAtKlo]

The curl command that the azure portal generates for it is: curl "https://myopenairesourcename.openai.azure.com/openai/deployments/thedeploymentnameIvegiven/chat/completions?api-version=2023-07-01-preview" \ -H "Content-Type: application/json" \ -H "api-key: YOUR_API_KEY" \ -d "{ \"messages\": [{\"role\":\"system\",\"content\":\"You are an AI assistant that helps people find information.\"}], \"max_tokens\": 800, \"temperature\": 0.7, \"frequency_penalty\": 0, \"presence_penalty\": 0, \"top_p\": 0.95, \"stop\": null }"

[kardolus]

Thanks! The settings below should match the provided curl. I am curious though if the response is parsed correctly. Could you give these settings a try? If they're not working, could you send me the response you are getting from Azure (from the curl you mentioned)? You can add the stream parameter too and use "stream": true and "stream": false. Hopefully those result in two different responses.

name: azure
api_key: <your api key>
model: gpt-4
max_tokens: 800
temperature: 0.7
top_p: 0.95
frequency_penalty: 0
presence_penalty: 0
url: https://myopenairesourcename.openai.azure.com/openai/deployments/thedeploymentnameIvegiven
completions_path: /chat/completions?api-version=2023-07-01-preview
auth_header: api-key
auth_token_prefix: ""

[kardolus]

This works (just tested it) :) -->

name: azure
api_key: <your_key>
model: <not relevant, read from the URL>
max_tokens: 4000
role: You are a helpful assistant.
temperature: 1
top_p: 1
frequency_penalty: 0
presence_penalty: 0
thread: default
omit_history: false
url: https://<your_resource>.openai.azure.com
completions_path: /openai/deployments/<your_deployment>/chat/completions?api-version=<your_api>
models_path: /v1/models
auth_header: api-key
auth_token_prefix: " "

[AlaShibanAtKlo]

thanks again! awesome work!