spring-projects / spring-ai

An Application Framework for AI Engineering
https://docs.spring.io/spring-ai/reference/index.html
Apache License 2.0
3.37k stars 869 forks source link

Add support for custom WebClientBuilder #739

Open fmunch opened 6 months ago

fmunch commented 6 months ago

This might be related to #512 and #609.

Expected Behavior

The *Api client classes should offer a way to customize the WebClient used. For example to add an Authorization header for custom Ollama setups, or to add some logging.

Current Behavior

Most *Api clients offer a constructor with a RestClient.Builder parameter, but also create a hardcoded WebClient.Builder:

public OllamaApi(String baseUrl, RestClient.Builder restClientBuilder) {
    // ...
    this.restClient = restClientBuilder.baseUrl(baseUrl).defaultHeaders(defaultHeaders).build();
    this.webClient = WebClient.builder().baseUrl(baseUrl).defaultHeaders(defaultHeaders).build();
}

public MistralAiApi(String baseUrl, String mistralAiApiKey, RestClient.Builder restClientBuilder,
        ResponseErrorHandler responseErrorHandler) {
    // ...
    this.restClient = restClientBuilder.baseUrl(baseUrl)
        .defaultHeaders(jsonContentHeaders)
        .defaultStatusHandler(responseErrorHandler)
        .build();
    this.webClient = WebClient.builder().baseUrl(baseUrl).defaultHeaders(jsonContentHeaders).build();
}

public OpenAiApi(String baseUrl, String openAiToken, RestClient.Builder restClientBuilder,
        ResponseErrorHandler responseErrorHandler) {
    this.restClient = restClientBuilder
            .baseUrl(baseUrl)
            .defaultHeaders(ApiUtils.getJsonContentHeaders(openAiToken))
            .defaultStatusHandler(responseErrorHandler)
            .build();
    this.webClient = WebClient.builder()
            .baseUrl(baseUrl)
            .defaultHeaders(ApiUtils.getJsonContentHeaders(openAiToken))
            .build();
}

This does not allow to customize the WebClient used in streaming methods.

Context

We want to deploy a Spring AI cluster that calls an Ollama instance, ideally with some sort of authentication.

markpollack commented 4 months ago

Sounds super reasonable. Will investigate.

ThomasVitale commented 1 month ago

Update: Support for custom WebClient.Builder has been added already for OpenAI and Ollama. It's still missing for Mistral AI.