Azure OpenAI : org-ai cannot handle api response from api version `2023-07-01-preview`, works fine with api version `2023-03-15-preview`

[doctorguile]

Hi

I have a patch for Azure OpenAI here,

https://github.com/doctorguile/org-ai/commit/e6ddc16889a7eb4575d404f026caced38c6fde17

but I haven't submitted the PR yet because it seems like org-ai cannot handle newer api response json format from openai

Is Emacs the greatest editor?
#+end_ai

api version 2023-07-01-preview response

{
  "id": "",
  "object": "",
  "created": 0,
  "model": "",
  "prompt_filter_results": [
    {
      "prompt_index": 0,
      "content_filter_results": {
        "hate": {
          "filtered": false,
          "severity": "safe"
        },
        "self_harm": {
          "filtered": false,
          "severity": "safe"
        },
        "sexual": {
          "filtered": false,
          "severity": "safe"
        },
        "violence": {
          "filtered": false,
          "severity": "safe"
        }
      }
    }
  ],
  "choices": []
}

{
  "id": "chatcmpl-8PNZz2f8dnCVFyLj0AFWVnBlp5bPq",
  "object": "chat.completion.chunk",
  "created": 1701059531,
  "model": "gpt-35-turbo-16k",
  "choices": [
    {
      "index": 0,
      "finish_reason": null,
      "delta": {
        "role": "assistant"
      },
      "content_filter_results": {}
    }
  ]
}

{
  "id": "chatcmpl-8PNZz2f8dnCVFyLj0AFWVnBlp5bPq",
  "object": "chat.completion.chunk",
  "created": 1701059531,
  "model": "gpt-35-turbo-16k",
  "choices": [
    {
      "index": 0,
      "finish_reason": null,
      "delta": {
        "content": "As"
      },
      "content_filter_results": {
        "hate": {
          "filtered": false,
          "severity": "safe"
        },
        "self_harm": {
          "filtered": false,
          "severity": "safe"
        },
        "sexual": {
          "filtered": false,
          "severity": "safe"
        },
        "violence": {
          "filtered": false,
          "severity": "safe"
        }
      }
    }
  ]
}

{
  "id": "chatcmpl-8PNZz2f8dnCVFyLj0AFWVnBlp5bPq",
  "object": "chat.completion.chunk",
  "created": 1701059531,
  "model": "gpt-35-turbo-16k",
  "choices": [
    {
      "index": 0,
      "finish_reason": null,
      "delta": {
        "content": " an"
      },
      "content_filter_results": {
        "hate": {
          "filtered": false,
          "severity": "safe"
        },
        "self_harm": {
          "filtered": false,
          "severity": "safe"
        },
        "sexual": {
          "filtered": false,
          "severity": "safe"
        },
        "violence": {
          "filtered": false,
          "severity": "safe"
        }
      }
    }
  ]
}

...

{
  "id": "chatcmpl-8PNZz2f8dnCVFyLj0AFWVnBlp5bPq",
  "object": "chat.completion.chunk",
  "created": 1701059531,
  "model": "gpt-35-turbo-16k",
  "choices": [
    {
      "index": 0,
      "finish_reason": null,
      "delta": {
        "content": "."
      },
      "content_filter_results": {
        "hate": {
          "filtered": false,
          "severity": "safe"
        },
        "self_harm": {
          "filtered": false,
          "severity": "safe"
        },
        "sexual": {
          "filtered": false,
          "severity": "safe"
        },
        "violence": {
          "filtered": false,
          "severity": "safe"
        }
      }
    }
  ]
}

{
  "id": "chatcmpl-8PNZz2f8dnCVFyLj0AFWVnBlp5bPq",
  "object": "chat.completion.chunk",
  "created": 1701059531,
  "model": "gpt-35-turbo-16k",
  "choices": [
    {
      "index": 0,
      "finish_reason": "stop",
      "delta": {},
      "content_filter_results": {}
    }
  ]
}

data: [DONE]

Note the first response, the array "choices": [] is empty

This breaks org-ai's parsing.

[doctorguile]

attached 2023-03-15-preview api version response: note the choices array is always non empty

{
  "id": "chatcmpl-8PbDDLVedJuLxct8SVxvSlp3UwTKn",
  "object": "chat.completion.chunk",
  "created": 1701111935,
  "model": "gpt-35-turbo-16k",
  "choices": [
    {
      "index": 0,
      "finish_reason": null,
      "delta": {
        "role": "assistant"
      }
    }
  ]
}

{
  "id": "chatcmpl-8PbDDLVedJuLxct8SVxvSlp3UwTKn",
  "object": "chat.completion.chunk",
  "created": 1701111935,
  "model": "gpt-35-turbo-16k",
  "choices": [
    {
      "index": 0,
      "finish_reason": null,
      "delta": {
        "content": "This"
      }
    }
  ]
}

...

{
  "id": "chatcmpl-8PbDDLVedJuLxct8SVxvSlp3UwTKn",
  "object": "chat.completion.chunk",
  "created": 1701111935,
  "model": "gpt-35-turbo-16k",
  "choices": [
    {
      "index": 0,
      "finish_reason": null,
      "delta": {
        "content": "."
      }
    }
  ]
}

{
  "id": "chatcmpl-8PbDDLVedJuLxct8SVxvSlp3UwTKn",
  "object": "chat.completion.chunk",
  "created": 1701111935,
  "model": "gpt-35-turbo-16k",
  "choices": [
    {
      "index": 0,
      "finish_reason": "stop",
      "delta": {}
    }
  ]
}

data: [DONE]

[doctorguile]

I have a patch for this fix

org-ai-openai.el

@@ -277,7 +277,7 @@ the response into."
               ;; insert text
               (if-let* ((choices (or (alist-get 'choices response)
                                      (plist-get response 'choices)))
-                        (choice (aref choices 0))
+                        (choice (and (arrayp choices) (> (length choices) 0) (aref choices 0)))
                         (delta (plist-get choice 'delta)))
                   (cond
                    ((plist-get delta 'role)

Will clean up the PR, rebase to latest commit and send PR your way (to support azure openai)

Thanks

[doctorguile]

created PR to support azure open-ai along with fixes for the above issue

https://github.com/rksm/org-ai/pull/93/files

[rksm]

Thank you very much!