spring-projects / spring-ai

An Application Framework for AI Engineering
https://docs.spring.io/spring-ai/reference/index.html
Apache License 2.0
3.32k stars 850 forks source link

ChatResponseMetadata does not have data of the retrieved documents from the vector store #1747

Open iAMSagar44 opened 5 days ago

iAMSagar44 commented 5 days ago

Bug description The map in ChatResponseMetadata does not contain details of the retrieved documents from the vector store. The QuestionAnswerAdvisor has details of the retrieved documents and would have set the ChatResponseMetadata (by adding the retrieved documents to this key - qa_retrieved_documents), but I am unable to retrieve this data from the ChatResponse.

Environment Spring AI versions - tried with both the 1.0.0-M3 and the 1.0.0-SNAPSHOT versions

Steps to reproduce Steps to reproduce the issue.

  1. Create a ChatClient as follows -
this.chatClient = chatBuilder
                                .defaultSystem(
                                                """
                                                                    You are a helpful assistant. Answer the questions based on the information provided to you by the user.
                                                                    Do not provide any information that is not relevant to the user's question.
                                                                    If you are unsure about the answer, you can ask the user for more information.
                                                                    Include the source of the information in your response at the end in a new line starting with "Source: ".
                                                                """)
                                .defaultAdvisors(new PromptChatMemoryAdvisor(chatMemory),
                                                new QuestionAnswerAdvisor(vectorStore, SearchRequest.defaults()),
                                                new SimpleLoggerAdvisor())
                                .build();
  1. Check the keys in the ChatResponse object and notice that it does not contain the key qa_retrieved_documents
StreamResponseSpec streamResponseSpec = this.chatClient.prompt()
                                .user(userRequest.message())
                                .advisors(a -> a
                                                .param(PromptChatMemoryAdvisor.CHAT_MEMORY_CONVERSATION_ID_KEY,
                                                                conversation.getId())
                                                .param(PromptChatMemoryAdvisor.CHAT_MEMORY_RETRIEVE_SIZE_KEY, 40))
                                .stream();

                streamResponseSpec.chatResponse().blockFirst().getMetadata().keySet().forEach(key -> {
                        logger.info("Key: {}", key);
                });

Expected behavior The ChatResponseMetadata should contain details of the retrieved documents. The application might need this data for further use cases (like retrieve the Document metadata and return it along with the Assistant Response to the calling application)