Encountering `400 Bad Request` error with Anthropic API using `ChatMessage`

[ginccc]

Hi there! Trying to use Anthropic with the regular dev.langchain4j.data.message.ChatMessage but it seems to be expecting dev.langchain4j.model.anthropic.AnthropicMessage.

I am trying to find a generic way of using various LLM and creating a dev.langchain4j.model.chat.ChatLanguageModel seems to be the way to go. For the OpenAI implementation it works fine, but for Anthropic it throws the following error..

Any idea what's wrong?

code:

var messages = new LinkedList<ChatMessage>();
var chatLanguageModel = getChatLanguageModel(task.type(), task.parameters());
var messageResponse = chatLanguageModel.generate(messages);

org.jboss.resteasy.reactive.ClientWebApplicationException: Received: 'Bad Request, status code 400' when invoking: Rest Client method: 'io.quarkiverse.langchain4j.anthropic.AnthropicRestApi#createMessage' at org.jboss.resteasy.reactive.client.impl.RestClientRequestContext.unwrapException(RestClientRequestContext.java:195) at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.handleException(AbstractResteasyReactiveContext.java:324) at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:175) at org.jboss.resteasy.reactive.client.impl.RestClientRequestContext$1.lambda$execute$0(RestClientRequestContext.java:314) at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:279) at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:261) at io.vertx.core.impl.ContextInternal.lambda$runOnContext$0(ContextInternal.java:59) at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:566) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ... 2 more Caused by: jakarta.ws.rs.WebApplicationException: Bad Request, status code 400 at io.quarkus.rest.client.reactive.runtime.DefaultMicroprofileRestClientExceptionMapper.toThrowable(DefaultMicroprofileRestClientExceptionMapper.java:19) at io.quarkus.rest.client.reactive.runtime.MicroProfileRestClientResponseFilter.filter(MicroProfileRestClientResponseFilter.java:52) at org.jboss.resteasy.reactive.client.handlers.ClientResponseFilterRestHandler.handle(ClientResponseFilterRestHandler.java:21) at org.jboss.resteasy.reactive.client.handlers.ClientResponseFilterRestHandler.handle(ClientResponseFilterRestHandler.java:10) at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.invokeHandler(AbstractResteasyReactiveContext.java:231) at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:147) ... 12 more

[geoand]

Hi,

I am not sure what exactly you are trying to do, but if I understand the requirement correctly, I am see that doing something like:

@Singleton
public class SomeClass  {
    @Inject
    ChatLanguageModel chatModel;

    public void someMethod() {
        var response = chatModel.generate(new UserMessage("Hello, how are you today?"));
    }
}

works just fine

[ginccc]

I have debugged it a bit further, here is a dump of the AnthropicCreateMessageRequest from the dev.langchain4j.model.anthropic.AnthropicChatModel#142

AnthropicCreateMessageRequest(model=claude-3-haiku-20240307, messages=[AnthropicMessage(role=ASSISTANT, content=[AnthropicTextContent(text=Hola)]), AnthropicMessage(role=USER, content=[AnthropicTextContent(text=hi)])], system=null, maxTokens=1024, stopSequences=null, stream=false, temperature=0.7, topP=null, topK=null, tools=null)

Once this request gets executed on that line, I am getting the above error..

Anything wrong with my Request object here?

[geoand]

Any chance you can attach a sample application that behaves as you describe so I can run it myself?

[ginccc]

Here you go..

Quarkus: 3.10.1 quarkus-langchain4j-anthropic: 0.13.1

GET http://localhost:8080/bot?msg=Hello

import dev.langchain4j.data.message.AiMessage;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.model.anthropic.AnthropicChatModel;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.QueryParam;

import static java.util.Arrays.*;

@Path("/bot")
public class ReproduceErrorAPI {

    @GET
    public String say(@QueryParam("msg") String message) {
        var builder = AnthropicChatModel.builder();

        //TODO insert api key here
        builder.apiKey("");

        AnthropicChatModel chatModel = builder.build();

        return chatModel.generate(new UserMessage(message)).content().text(); // this works
        // return chatModel.generate(new AiMessage("Hi")).content().text(); // does not work
        // return chatModel.generate(new AiMessage("Hi"), new UserMessage(message)).content().text(); // does not work
        // return chatModel.generate(new UserMessage("Hi"), new UserMessage(message)).content().text(); // does not work
    }
}

[geoand]

Try adding:

        builder.logRequests(true);
        builder.logResponses(true);

and see what the response says.

[ginccc]

Thanks a lot for sparring this issue with me! This was the hint I needed..

Claude doesn't allow AiMessage to be first, it has to be UserMessage!

[geoand]

💪