Easy way to log requests and responses to LLM's

[pax95]

Was looking for ways to log request/response from call to LLM's, and found this discussion. I think it would be great if this would be supported as a configuration property as it is in Langchain4j.

Discussed in https://github.com/spring-projects/spring-ai/discussions/450

^{Originally posted by **iAMSagar44** March 15, 2024} Has anyone managed to find a way to log the requests and responses to/from Open AI using Spring AI. I have tried various logging settings in the application.properties but no luck. With Langchain4J, there is a specific property for logging the requests and responses to/from the AI models, but couldn't find anything similar with Spring AI.

[piotrooo]

I handle this using the Logbook logging library.

@Bean
public RestClientCustomizer restClientCustomizer(Logbook logbook) {
    return restClientBuilder -> restClientBuilder.requestInterceptor(new LogbookClientHttpRequestInterceptor(logbook));
}

[pax95]

@piotrooo Thanks for the tip. IMHO it is's maybe a bit to much to introduce such a new dependency just for request/response logging.

[ThomasVitale]

I opened a discussion in https://github.com/spring-projects/spring-ai/issues/512 about this topic. In the meantime, this is how I solved it. You can see the full examples here: https://github.com/ThomasVitale/concerto-for-java-and-ai/blob/main/mousike/src/main/java/com/thomasvitale/mousike/ai/clients/HttpClientAutoConfiguration.java#L16

    @Bean
    RestClientCustomizer restClientCustomizer(HttpClientProperties httpClientProperties) {
        HttpClientConfig clientConfig = HttpClientConfig.builder()
                .connectTimeout(httpClientProperties.getConnectTimeout())
                .readTimeout(httpClientProperties.getReadTimeout())
                .sslBundle(httpClientProperties.getSslBundle())
                .logRequests(httpClientProperties.isLogRequests())
                .logResponses(httpClientProperties.isLogResponses())
                .build();

        return restClientBuilder -> {
            restClientBuilder
                    .requestFactory(new BufferingClientHttpRequestFactory(
                            ClientHttpRequestFactories.get(ClientHttpRequestFactorySettings.DEFAULTS
                                            .withConnectTimeout(clientConfig.connectTimeout())
                                            .withReadTimeout(clientConfig.readTimeout()))))
                    .requestInterceptors(interceptors -> {
                        if (clientConfig.logRequests() || clientConfig.logResponses()) {
                            interceptors.add(new HttpLoggingInterceptor(clientConfig.logRequests(), clientConfig.logResponses()));
                        }
                    });
        };
    }

[piotrooo]

@piotrooo Thanks for the tip. IMHO it is's maybe a bit to much to introduce such a new dependency just for request/response logging.

As always, it depends. If you want something for development purposes, a @ThomasVitale example is enough. On the other hand, if you want a fully-featured sample with configuration and battle-proven features, use library (e.g. logbook).

[pax95]

Both solutions are viable solutions for something that you would need in the initial development and debug fase. (I suspect that both solutions would affect other restclients too if you have Functions that uses a custom restClient) IMHO this should be easier for developers to enable using a configuration property as part of spring-ai.

[markpollack]

I've added some thoughts for discussion here. Certainly improvements in this area need to be made.

[ThomasVitale]

I shared some additional thoughts in https://github.com/spring-projects/spring-ai/issues/512#issuecomment-2185096414