search

PabloSanchi / IBM-WatsonxAI-Spring-AI-Example

Example of IBM watsonx.ai with Spring AI
https://docs.spring.io/spring-ai/reference/1.0-SNAPSHOT/api/chat/watsonx-ai-chat.html
1 stars 2 forks source link

[Question] how to chat continuously with watsonx.ai within context #1

Closed uniquejava closed 3 months ago

uniquejava commented 3 months ago

Hello Pablo,

Thank you for letting us use watsonx.ai API with ease via Spring AI. I integrated it successfully in our project. However, it cannot chat with us continuously within current context.

For example, if I select model "meta-llama/llama-3-70b-instruct`, asked a first question, it answered, and then I ask "What did I just ask you?", it just stopped. It lost the context.

Do I miss something. What additional components do we need?

My configuration:

spring:
  ai:
    watsonx:
      ai:
        base-url: https://jp-tok.ml.cloud.ibm.com
        stream-endpoint: /ml/v1/text/generation_stream?version=2024-03-13
        text-endpoint: /ml/v1/text/generation?version=2024-03-13
        project-id: xxxx
        iam-token: xxxx
PabloSanchi commented 3 months ago

Hi @uniquejava, the model it self does not keep track of the previous messages. The developer is the one that needs to take care. You can go to the spring ai docs to see how it is done :)

uniquejava commented 3 months ago

Hi @uniquejava, the model it self does not keep track of the previous messages. The developer is the one that needs to take care. You can go to the spring ai docs to see how it is done :)

Thank you for your reply. Yes, I found ChatClient and ChatMemory, and I think that's what I need.