[Azure OpenAI] Function calling - Can't execute multiple function calls

[tha2015]

Bug description When there should be multiple function calls for a single question, only one call is made because AzureOpenAiChatClient doesn't copy tools value from first request to second request, as a result, the next function call will not happen (loosing context in the function call response).

Environment spring-ai-azure-openai-1.0.0-20240420.063719-112.jar

Steps to reproduce

• AzureOpenAiChatClient makes a call to gpt-4 with question "What's the weather like in San Francisco, Tokyo, and Paris?" and function "weatherFunction"
• AzureOpenAiChatClient receives a function call in response for "San Francisco"
• AzureOpenAiChatClient calls the function and builds a second request including the result, sends new request
• AzureOpenAiChatClient receives the result for "San Francisco" and stops

Expected behavior Multiple function calls should happen (for SanFrancisco, Tokyo, Paris).

Analysis This issue doesn't happen if using OpenAIChatClient. And comparing OpenAiChatClient.doCreateToolResponseRequest and AzureOpenAiChatClient.doCreateToolResponseRequest shows that OpenAiChatClient uses ModelOptionsUtils.merge(newRequest, previousRequest, ChatCompletionRequest.class) to copy attributes (including tools) from first request to second request, while AzureOpenAiChatClient uses AzureOpenAiChatClient.merge to copy attributes (missing tools). So it looks like missing tools property is causing the issue.

Minimal Complete Reproducible example

   // Kotlin code snippet
    val response: ChatResponse = chatClient.call(
        Prompt(
            listOf(UserMessage("What's the weather like in San Francisco, Tokyo, and Paris?")),
            AzureOpenAiChatOptions.builder().withFunction("weatherFunction").build()
        )
    )

    println(response.result.output.content)

Getting weather for San Francisco in C
The current temperature in San Francisco is 30 degrees Celsius. 

Now let's check the weather in Tokyo and Paris. 

[tzolov]

@tha2015 our integration tests for parallel function calling are successful: https://github.com/spring-projects/spring-ai/tree/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/azure/tool

Mind that you need to deploy a mode gpt-4-0125-preview or new.

[tha2015]

Thanks Christian. I could not set up spring-ai project to run the tests (there were some compile errors) and also I don't have an Azure key to test it. Actually I'm using AzureOpenAiChatClient with some custom logic for my environment so flows might be a bit different and I can't explain why the IT tests work and my logic doesn't. But I still suspect that AzureOpenAiChatClient.doCreateToolResponseRequest has some issue because it doesn't copy tools from the previous request to the next request. Can you have a look when you have some time? Thanks.

[tha2015]

@tzolov I believe that my model is not latest so it doesn't support Parallel function calling. As a result, it requires 3 round trips to do 3 function calls (instead of one). And your test seems to work with newer model and just requires one round trip to do 3 function calls. When there are multiple round trips, the tools value must be copied from first request to the following requests but it didn't happen. Can you review logic of AzureOpenAiChatClient.doCreateToolResponseRequest to see why it doesn't copy tools property? Thanks.

[avani17101]

@tzolov I am facing this for gpt4o too. I assume it is definitely newer than gpt-4-0125-preview

[timostark]

@tha2015 @avani17101 raised a PR https://github.com/spring-projects/spring-ai/pull/1042 .. let's see if it can get merged that way.