Support for Azure OpenAI seems broken

[mads-oestergaard]

Seems like support for Azure OpenAI broke.

As far as I can see, the engine config is no longer parsed and used in the code anywhere, as it were earlier (https://github.com/j178/chatgpt/pull/58).

It looks like the engine parameter is needed on openai.ChatCompletion instead of model when using AzureOpenAI, as per this Azure example:

completion = openai.ChatCompletion.create(
  engine="my-deployment",
  messages = message_text,
  temperature=0.7,
  max_tokens=800,
  top_p=0.95,
  frequency_penalty=0,
  presence_penalty=0,
  stop=None
)

[j178]

Hi, thanks for the report!

Unfortunately, I don't have an access to Azure OpenAI. Could you help testing #82 and check if it's working? Thanks a lot!

To run the code on that branch, run

go install github.com/j178/chatgpt/cmd/chatgpt@fix-azure

and configure a model name to deployment mapping:

{
  "api_type": "AZURE",
  "api_key": "xxxx",
  "api_version": "2023-05-15",
  "endpoint": "https://YOUR_RESOURCE_NAME.openai.azure.com",
  "model_mapping": {
    "gpt-3.5-turbo": "your gpt-3.5-turbo deployment name",
    "gpt-4": "your gpt-4 deployment name"
  }
}

[j178]

Hi, I released v1.3.3 with the fix, could you try it?

[mads-oestergaard]

Yeah, just tried it, I get the following crash, happens just as I press a key.

$ chatgpt
Caught panic:

runtime error: invalid memory address or nil pointer dereference

Restoring terminal...

goroutine 1 [running]:
runtime/debug.Stack()
        runtime/debug/stack.go:24 +0x5e
runtime/debug.PrintStack()
        runtime/debug/stack.go:16 +0x13
github.com/charmbracelet/bubbletea.(*Program).Run.func1()
        github.com/charmbracelet/bubbletea@v0.25.0/tea.go:478 +0x91
panic({0xe1ae00?, 0x1612b60?})
        runtime/panic.go:920 +0x270
github.com/pkoukk/tiktoken-go.(*Tiktoken).Encode(0x0, {0xea3444, 0x6}, {0x0?, 0x1?, 0x10?}, {0x0, 0x0, 0xc00006e180?})
        github.com/pkoukk/tiktoken-go@v0.1.6/tiktoken.go:81 +0x372
github.com/j178/chatgpt/tokenizer.CountTokens({0xc00009d007, 0x3}, {0xea3444, 0x6})
        github.com/j178/chatgpt/tokenizer/tokenize.go:16 +0xd0
github.com/j178/chatgpt/tokenizer.CountMessagesTokens({0xc00009d007, 0x3}, {0xc0004c6000, 0x1, 0x8?})
        github.com/j178/chatgpt/tokenizer/tokenize.go:37 +0x1ad
github.com/j178/chatgpt.(*Conversation).GetContextTokens(0xc000340080)
        github.com/j178/chatgpt/conversation.go:222 +0x49
github.com/j178/chatgpt/ui.Model.RenderFooter({0x77, 0x1e, 0x0, 0x0, {0x0, 0x0}, {{{0xc0004704a0, 0x1, 0x1}, {{...}, ...}, ...}, ...}, ...})
        github.com/j178/chatgpt/ui/ui.go:430 +0x867
github.com/j178/chatgpt/ui.Model.View({0x77, 0x1e, 0x0, 0x0, {0x0, 0x0}, {{{0xc0004704a0, 0x1, 0x1}, {{...}, ...}, ...}, ...}, ...})
        github.com/j178/chatgpt/ui/ui.go:476 +0xd6
github.com/charmbracelet/bubbletea.(*Program).eventLoop(0xc00058ed00, {0x10ed750?, 0xc0000bd980?}, 0xc00057c4a8?)
        github.com/charmbracelet/bubbletea@v0.25.0/tea.go:413 +0x798
github.com/charmbracelet/bubbletea.(*Program).Run(0xc00058ed00)
        github.com/charmbracelet/bubbletea@v0.25.0/tea.go:543 +0x88e
main.main()
        github.com/j178/chatgpt/cmd/chatgpt/main.go:124 +0x849

[j178]

Hi, what's the conversation.model value in your configuration?

[mads-oestergaard]

This is my configuration, or parts of it anyway

{
  "api_key": "",
  "endpoint": "https://XXXX.openai.azure.com/",
  "api_type": "AZURE",
  "api_version": "2023-07-01-preview",
  "model_mapping": {
    "gpt-3.5-turbo": "turbo-engine",
    "gpt-4": "gp4-engine"
  },
  "conversation": {
    "prompt": "default",
    "context_length": 6,
    "model": "gpt-4",
    "stream": true,
    "temperature": 0,
    "max_tokens": 1024
  },

[j178]

Thanks. Could you try v1.3.4 to see if the issue resolved?

[mads-oestergaard]

Works now, thanks!