[FEATURE REQ] Improve/document function calling with streaming

[rstropek]

Library name

Azure.AI.OpenAI

Please describe the feature.

When combining function calling with streaming, the behavior of OpenAI/this library is not apparent. Here is an example:

using Azure.AI.OpenAI;
using Azure;
using System.Text.Json;

var oaiKey = "<add your key>";
var oaiEndpoint = "https://<add your endpoint>.openai.azure.com/";
var oai = new OpenAIClient(new Uri(oaiEndpoint), new AzureKeyCredential(oaiKey));

// Set test scenario
var options = new ChatCompletionsOptions()
{
    Messages =
    {
        new(ChatRole.System, "You are a helpful assistent. You have to find out the age of the user."),
        new(ChatRole.User, "Hi!"),
        new(ChatRole.Assistant, "Hello! Can you tell me your age?"),
        new(ChatRole.User, "I am 25 years old."),
    },
    Functions =
    {
        new()
        {
            Name = "set_age",
            Description = "Sets the age of the user.",
            Parameters = BinaryData.FromObjectAsJson(new
                {
                    Type = "object",
                    Properties = new
                    {
                        Age = new
                        {
                            Type = "integer",
                            Description = "The age of the user"
                        },
                    },
                    Required = new[] { "age" }
                },
                new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }
            )
        },
    }
};

var streamingResponse = await oai.GetChatCompletionsStreamingAsync("<your-deployment>", options);
await foreach (var choices in streamingResponse.Value.GetChoicesStreaming())
{
    await foreach (var streamingChoiceMessage in choices.GetMessageStreaming())
    {
        Console.WriteLine(JsonSerializer.Serialize(streamingChoiceMessage));
    }
}

The output is:

{"Role":{},"Content":null,"Name":null,"FunctionCall":{"Name":"set_age","Arguments":null},"AzureExtensionsContext":null}
{"Role":{},"Content":null,"Name":null,"FunctionCall":{"Name":null,"Arguments":"{\n"},"AzureExtensionsContext":null}
{"Role":{},"Content":null,"Name":null,"FunctionCall":{"Name":null,"Arguments":" "},"AzureExtensionsContext":null}
{"Role":{},"Content":null,"Name":null,"FunctionCall":{"Name":null,"Arguments":" \u0022"},"AzureExtensionsContext":null}
{"Role":{},"Content":null,"Name":null,"FunctionCall":{"Name":null,"Arguments":"age"},"AzureExtensionsContext":null}
{"Role":{},"Content":null,"Name":null,"FunctionCall":{"Name":null,"Arguments":"\u0022:"},"AzureExtensionsContext":null}
{"Role":{},"Content":null,"Name":null,"FunctionCall":{"Name":null,"Arguments":" "},"AzureExtensionsContext":null}
{"Role":{},"Content":null,"Name":null,"FunctionCall":{"Name":null,"Arguments":"25"},"AzureExtensionsContext":null}
{"Role":{},"Content":null,"Name":null,"FunctionCall":{"Name":null,"Arguments":"\n"},"AzureExtensionsContext":null}
{"Role":{},"Content":null,"Name":null,"FunctionCall":{"Name":null,"Arguments":"}"},"AzureExtensionsContext":null}
{"Role":{},"Content":null,"Name":null,"FunctionCall":null,"AzureExtensionsContext":null}

In order to process a function call, one needs to "stitch together" the results. Otherwise, a caller never receives the function's name and arguments in a single response. I am aware that this library behavior reflects the OpenAI service's behavior. So it is not wrong. However, at least for me it was not clear from reading the docs.

It would be nice if the library would contain specific support for that scenario. If that is not possible, having a sample or a note in the docs would be helpful.

[rstropek]

BTW - if I can help enhance the docs, please point me in the right direction. I would be happy to help.

[github-actions[bot]]

Thank you for your feedback. This has been routed to the support team for assistance.

[navba-MSFT]

Adding service team to look into this.