spring-projects / spring-ai

An Application Framework for AI Engineering
https://docs.spring.io/spring-ai/reference/1.0-SNAPSHOT/index.html
Apache License 2.0
2.48k stars 588 forks source link

Add request/response logging in OpenAiApi #908

Open markpollack opened 1 week ago

markpollack commented 1 week ago

For the chat models and the embedding models that are in OpenAiApi add logging at the debug level.

This should also be done for streaming endpoints using the technique in PromptChatMemoryAdvisor, e.g.


    @Override
    public Flux<ChatResponse> adviseResponse(Flux<ChatResponse> fluxChatResponse, Map<String, Object> context) {

        return new MessageAggregator().aggregate(fluxChatResponse, chatResponse -> {
            List<Message> assistantMessages = chatResponse.getResults()
                .stream()
                .map(g -> (Message) g.getOutput())
                .toList();

            this.getChatMemoryStore().add(this.doGetConversationId(context), assistantMessages);
        });
    }
ThomasVitale commented 1 week ago

I shared some thoughts about this in https://github.com/spring-projects/spring-ai/issues/512#issuecomment-2185096414