[BUG] NullPointerException: Cannot invoke "com.azure.ai.openai.models.ChatResponseMessage.getContent()" because "delta" is null

[leduckhc]

Hi,

this seems to affect AzureOpenAiStreamingChatModel. I am hypothesising that LocalAiStreamingChatModel since the code that handle response code seems to be similar to AzureOpenAiStreamingChatModel

Describe the bug

Using AiService on streaming models

Log and Stack trace

Caused by: java.lang.NullPointerException: Cannot invoke "com.azure.ai.openai.models.ChatResponseMessage.getContent()" because "delta" is null

To Reproduce

interface Assistant {
    TokenStream chat(String message);
}

StreamingChatLanguageModel model = AzureOpenAiStreamingChatModel.builder()
        .endpoint(AZURE_ENDPOINT)
        .apiKey(AZURE_API_KEY)
        .deploymentName(deploymentName)
        .useAsyncClient(true)
        .tokenizer(new AzureOpenAiTokenizer(deploymentName))
        .logRequestsAndResponses(true)
        .build();

TokenStream tokens = AiServices.builder(Assistant.class)
        .streamingChatLanguageModel(model)
        .build()
        .chat("Tell me a joke about Python developers");

Expected behavior

Please complete the following information:

• LangChain4j version: 0.32.0, 033.0, 0.34.0
• LLM(s) used: AzureOpenAiStreamingChatModel
• Java version: 20
• Spring Boot version (if applicable):

Additional context

[langchain4j-github-bot[bot]]

/cc @agoncal (azure), @jdubois (azure)

[langchain4j]

Hi @leduckhc, which LLM provider are you using?

[leduckhc]

Sorry, for confusion - it's AzureOpenAiStreamingChatModel problem so far - I have updated the issue description.

Currently the problem seems to arise on AzureOpenAi provider. I am hypothesising that the same error might occur with LocalAi provider - since the code that handle the response message seems to be similar to AzureOpenAi.

[langchain4j]

I see. I guess https://github.com/langchain4j/langchain4j/pull/1720 fixes it?

[leduckhc]

1720 seems to be the fix. Thanks