[Bug]: Undefined array key "message" (Azure with Vector Database)

DC-Sebastian commented 5 months ago

Description

I am trying to use Azure in conjunction with a vector database

According to Microsoft's documentation, the endpoint looks like this:

YOUR_RESOURCE_NAME/openai/deployments/YOUR_DEPLOYMENT_NAME/extensions/chat/completions?api-version=2023-06-01-preview

{
    "id": "12345678-1a2b-3c4e5f-a123-12345678abcd",
    "model": "",
    "created": 1684304924,
    "object": "chat.completion",
    "choices": [
        {
            "index": 0,
            "messages": [
                {
                    "role": "tool",
                    "content": "{\"citations\": [{\"content\": \"\\nAzure AI services are cloud-based artificial intelligence (AI) services...\", \"id\": null, \"title\": \"What is Azure AI services\", \"filepath\": null, \"url\": null, \"metadata\": {\"chunking\": \"orignal document size=250. Scores=0.4314117431640625 and 1.72564697265625.Org Highlight count=4.\"}, \"chunk_id\": \"0\"}], \"intent\": \"[\\\"Learn about Azure AI services.\\\"]\"}",
                    "end_turn": false
                },
                {
                    "role": "assistant",
                    "content": " \nAzure AI services are cloud-based artificial intelligence (AI) services that help developers build cognitive intelligence into applications without having direct AI or data science skills or knowledge. [doc1]. Azure Machine Learning is a cloud service for accelerating and managing the machine learning project lifecycle. [doc1].",
                    "end_turn": true
                }
            ]
        }
    ]
}

The factory code for openai-php should look like this

 $client = OpenAI::factory()
      ->withBaseUri('{your-resource-name}.openai.azure.com/openai/deployments/{deployment-id}/extensions/')
      ->withHttpHeader('api-key', {your-api-key})
      ->withQueryParam('api-version', '2023-06-01-preview')
      ->make();

    $result = $client->chat()->create([
      'model' => "{deployment-id}"
      "dataSources" => [
        [
          "type" => "AzureCognitiveSearch",
          "parameters" => [
            "endpoint" => "YOUR_AZURE_COGNITIVE_SEARCH_ENDPOINT",
            "key" => "YOUR_AZURE_COGNITIVE_SEARCH_KEY",
            "indexName" => "YOUR_AZURE_COGNITIVE_SEARCH_INDEX_NAME"
          ]
        ]
      ],
      'messages' => [
        ['role' => 'user', 'content' => 'What are the differences between Azure Machine Learning and Azure AI services?'],
      ],
    ]);

However, I get the error message Undefined array key "message"

Is the reason for the error message that the message contains a "role": "tool" with the data from the vector database?

"messages": [
    {
        "role": "tool",
        "content": "{\"citations\": [{\"content\": \"\\nAzure AI services are cloud-based artificial intelligence (AI) services...\", \"id\": null, \"title\": \"What is Azure AI services\", \"filepath\": null, \"url\": null, \"metadata\": {\"chunking\": \"orignal document size=250. Scores=0.4314117431640625 and 1.72564697265625.Org Highlight count=4.\"}, \"chunk_id\": \"0\"}], \"intent\": \"[\\\"Learn about Azure AI services.\\\"]\"}",
        "end_turn": false
    },
    {
        "role": "assistant",
        "content": " \nAzure AI services are cloud-based artificial intelligence (AI) services that help developers build cognitive intelligence into applications without having direct AI or data science skills or knowledge. [doc1]. Azure Machine Learning is a cloud service for accelerating and managing the machine learning project lifecycle. [doc1].",
        "end_turn": true
    }
]

If this is the problem, is it possible to add the "role" : "tool" to the library, the rest of the response corresponds to the format of open ai.

Steps To Reproduce

See Description.

OpenAI PHP Client Version

v0.8.1

PHP Version

8.2.1

Notes

No response

DC-Sebastian commented 5 months ago

After looking again at what a response from openai looks like, I realize why it doesn't work

Openai outputs the response with message while Azure with a vector database outputs the response with messages.

OpenAI response example:

  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "\n\nHello there, how may I assist you today?",
    },
    "logprobs": null,
    "finish_reason": "stop"
  }],

Azure response example:

   "choices": [
        {
            "index": 0,
            "messages": [
                {
                    "role": "tool",
                    "content": "{\"citations\": [{\"content\": \"\\nAzure AI services are cloud-based artificial intelligence (AI) services...\", \"id\": null, \"title\": \"What is Azure AI services\", \"filepath\": null, \"url\": null, \"metadata\": {\"chunking\": \"orignal document size=250. Scores=0.4314117431640625 and 1.72564697265625.Org Highlight count=4.\"}, \"chunk_id\": \"0\"}], \"intent\": \"[\\\"Learn about Azure AI services.\\\"]\"}",
                    "end_turn": false
                },
                {
                    "role": "assistant",
                    "content": " \nAzure AI services are cloud-based artificial intelligence (AI) services that help developers build cognitive intelligence into applications without having direct AI or data science skills or knowledge. [doc1]. Azure Machine Learning is a cloud service for accelerating and managing the machine learning project lifecycle. [doc1].",
                    "end_turn": true
                }
            ]
        }
    ]

Here is another example of what Azure's streaming response looks like:

data:{"id": "", "model": "", "created": 1705234910, "object": "chat.completion.chunk", "choices": [{"index": 0, "messages": [{"delta": {"role": "tool", "content": "matched vector database content"}, "index": 0, "end_turn": false}], "finish_reason": null}]}

data:{"id": "", "model": "", "created": 1705234910, "object": "chat.completion.chunk", "choices": [{"index": 0, "messages": [{"delta": {"role": "assistant"}, "index": 1, "end_turn": false}], "finish_reason": null}]}

data:{"id": "", "model": "", "created": 1705234910, "object": "chat.completion.chunk", "choices": [{"index": 0, "messages": [{"delta": {"content": "text"}, "index": 1, "end_turn": false}], "finish_reason": null}]}

data:{"id": "", "model": "", "created": 1705234910, "object": "chat.completion.chunk", "choices": [{"index": 0, "messages": [{"delta": {"content": "text"}, "index": 1, "end_turn": false}], "finish_reason": null}]}

data:{"id": "", "model": "", "created": 1705234910, "object": "chat.completion.chunk", "choices": [{"index": 0, "messages": [{"delta": {"content": "[DONE]"}, "index": 2, "end_turn": true}], "finish_reason": null}]}

Is it possible to adapt openai-php so that messages can also be processed?

gehrisandro commented 5 months ago

HI @DC-Sebastian

Thanks for the comprehensive explanation.

I am not sure atm if we should fully support Azure as the API seem to diverge more and more.

I will talk about that with @nunomaduro

DC-Sebastian commented 5 months ago

Thank you for your answer. Of course, I can understand if you can't support the new functionalities of Azure, supporting two APIs that evolve differently is a lot of extra work.

jhuhta commented 4 months ago

Have you decided on the Azure support? Currently, it seems it's impossible to use this library for chat/completions because the response message structure is different. However, this one would be a small thing to address still, but obviously I don't know about the possible next updates.

openai-php / client