Bug / Patch: System instructions are lost in VertexAiGeminiChatModel

[turchinc]

Bug description

The system instructions I was providing indicated a JSON output format, but were getting ignored in Spring AI. The same exact instructions and prompt in the test console work perfectly.

https://console.cloud.google.com/vertex-ai/generative/multimodal/create/text

I debugged into this and even thought I could put in a PR for it, but honestly, I don't see where this goes wrong. In private GeminiRequest createGeminiRequest(Prompt prompt) I can clearly see my system instructions and the code looks like it adds them to the model:

              String systemContext = prompt.getInstructions()
            .stream()
            .filter(m -> m.getMessageType() == MessageType.SYSTEM)
            .map(m -> m.getContent())
            .collect(Collectors.joining(System.lineSeparator()));

        if (StringUtils.hasText(systemContext)) {
            generativeModel.withSystemInstruction(ContentMaker.fromString(systemContext));
        }

        return new GeminiRequest(toGeminiContent(prompt), generativeModel);

But in the debugger, at the return statement, the system instructions are empty:

Environment Please provide as many details as possible: Spring AI version, Java version, which vector store you use if any, etc

    implementation platform("org.springframework.ai:spring-ai-bom:1.0.0-SNAPSHOT")
    implementation 'org.springframework.ai:spring-ai-vertex-ai-gemini'
    implementation 'org.springframework.ai:spring-ai-vertex-ai-embedding-spring-boot-starter'

I should note, until this morning the code did not work at all, but the PR from last night solved that :)

Steps to reproduce

See code below.

Expected behavior

The system instructions are passed along in the gemini request.

Minimal Complete Reproducible example

package my.rag;

import org.springframework.ai.chat.messages.Message;
import org.springframework.ai.chat.messages.SystemMessage;
import org.springframework.ai.chat.messages.UserMessage;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.model.Generation;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.vertexai.gemini.VertexAiGeminiChatModel;
import org.springframework.ai.vertexai.gemini.VertexAiGeminiChatOptions;

import com.google.cloud.vertexai.VertexAI;

import java.util.ArrayList; 

public class Test {
    public static void main(String[] args) {
                var chatModel = new VertexAiGeminiChatModel(
                new VertexAI(Config.GOOGLE_CLOUD_PROJECT_ID, Config.GOOGLE_GEMINI_API_LOCATION),
                VertexAiGeminiChatOptions.builder()
                        .withModel(VertexAiGeminiChatModel.ChatModel.GEMINI_1_5_FLASH)
                        .withTemperature(1f)
                        .build());
        var messages = new ArrayList<Message>();
        var systemMessage = new StringBuilder();
        systemMessage.append("You are a helpful assistant, providing guidance based on the information provided below. Always return as JSON format:\n")
        .append("Format example:\n")
        .append("{\"response\": \"This is the useful response from the helpful assistant\", \"link\": \"https://link-for-more-information\"}");
        System.out.println(systemMessage.toString());
        messages.add(new SystemMessage(systemMessage.toString()));
        var userMessage = "What is the URL for the google search engine?";
        messages.add(new UserMessage( userMessage));
        System.out.println(userMessage);

        ChatResponse response = chatModel.call(new Prompt(messages));
        for (Generation aiResult : response.getResults()) {
            System.out.println(aiResult.getOutput().getContent());
        }
    }
}

Running this code the RAG responds with something to the effect:

The URL for the Google search engine is simply:

**https://www.google.com**

In the console I get

{"response": "The URL for the Google search engine is www.google.com", "link": "https://www.google.com"}

[turchinc]

I see it now, this PR should resolve it: the withSystemInstruction(...) is a fluent API and returns a new model.

https://github.com/spring-projects/spring-ai/pull/1031

[tzolov]

Thank you @turchinc for catching and resolving this issue! Highly appreciated!

[turchinc]

@tzolov thanks for applying the fix! I can confirm my app is working now with snapshot.