Cannot get the service to return anything but a `String`.

FroMage commented 5 months ago

The docs at https://docs.quarkiverse.io/quarkus-langchain4j/dev/ai-services.html#_ai_method_return_type say I should be able to get a structured return type from my service, but it does not seem to support anything but a String.

package util;

import java.util.List;

import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;
import io.quarkiverse.langchain4j.RegisterAiService;

@RegisterAiService( 
        retrievalAugmentor = ScheduleDocumentRetreiver.class
)
public interface ScheduleAI {

    @SystemMessage("You are a computer science conference organiser") 
    @UserMessage("""
                I want to find the talks from the conference program that match my interests and constraints.
                Give me the list of talks that match my interests and constraints: {topics}
            """)
    List<AITalk> findTalks(String topics);

    public static class AITalk {
        public String title;
        public String id;
    }
}

package util;

import java.util.function.Supplier;

import dev.langchain4j.model.embedding.EmbeddingModel;
import dev.langchain4j.rag.DefaultRetrievalAugmentor;
import dev.langchain4j.rag.RetrievalAugmentor;
import dev.langchain4j.rag.content.retriever.EmbeddingStoreContentRetriever;
import io.quarkiverse.langchain4j.pgvector.PgVectorEmbeddingStore;
import jakarta.inject.Singleton;

@Singleton
public class ScheduleDocumentRetreiver implements Supplier<RetrievalAugmentor> {

    private final RetrievalAugmentor augmentor;

    ScheduleDocumentRetreiver(PgVectorEmbeddingStore store, EmbeddingModel model) {
        EmbeddingStoreContentRetriever contentRetriever = EmbeddingStoreContentRetriever.builder()
                .embeddingModel(model)
                .embeddingStore(store)
                .maxResults(3)
                .build();
        augmentor = DefaultRetrievalAugmentor
                .builder()
                .contentRetriever(contentRetriever)
                .build();
    }

    @Override
    public RetrievalAugmentor get() {
        return augmentor;
    }

}

@Path("/")
@Blocking
public class Application {

    @CheckedTemplate
    public static class Templates {
        public static native TemplateInstance ai(String results);
    }

    @Inject
    ScheduleAI ai;

    @GET
    @Path("/ai")
    public TemplateInstance ai(@RestQuery String topics){
        StringBuilder results = new StringBuilder();
        if(topics != null && !topics.isBlank()) {
            results.append("Results: ");
            for (AITalk aiTalk : ai.findTalks(topics)) { // CCE EXCEPTION HERE
                results.append(aiTalk.id);
                results.append(", ");
            }
        }
        return Templates.ai(results.toString());
    }
}

This generates the following exception:

2024-06-13 12:07:03,893 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (executor-thread-1) HTTP Request to /ai?csrf-token=dQpPok314ThUaiceewld7g&topics=I+want+to+learn+about+hibernate%2C+and+web+applications failed, error id: a57d74f1-f0a0-4176-a120-25be5321f0d9-1: java.lang.ClassCastException: class java.lang.String cannot be cast to class util.ScheduleAI$AITalk (java.lang.String is in module java.base of loader 'bootstrap'; util.ScheduleAI$AITalk is in unnamed module of loader io.quarkus.bootstrap.classloading.QuarkusClassLoader @49cb1baf)
    at rest.Application.ai(Application.java:518)
    at rest.Application_ClientProxy.ai(Unknown Source)
    at rest.Application$quarkusrestinvoker$ai_c38fa7a6cc502cf9089d9e3dafbbfabcc91df15a.invoke(Unknown Source)
    at org.jboss.resteasy.reactive.server.handlers.InvocationHandler.handle(InvocationHandler.java:29)
    at io.quarkus.resteasy.reactive.server.runtime.QuarkusResteasyReactiveRequestContext.invokeHandler(QuarkusResteasyReactiveRequestContext.java:141)
    at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:147)
    at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:599)
    at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
    at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
    at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:11)
    at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:11)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:1583)

So it looks like the generated service interface has String as a method return type.

Also, @geoand told me:

First of all, try returning a domain object, instead of a string that will result in the prompt being augmented with the proper instructions on how to return the result

But the docs say:

If the prompt defines the JSON response format precisely, you can map the response directly to an object:

So that's contradictory.