Tool executions on streaming models lead to `The current thread cannot be blocked: vert.x-eventloop-thread-0`

[dastrobu]

When combining a streaming model with a tool, that does e.g. a rest call, it leads to the following error message:

The current thread cannot be blocked: vert.x-eventloop-thread-0

Here is an example:

    @Singleton
    static class MyTool {

        @Tool(value = "the answer to life, the universe and everything")
        public String answer() {
            return Uni.createFrom().item("42")
                    .onItem().delayIt().by(Duration.ofMillis(500))
                    .await().indefinitely();
        }
    }

    @RegisterAiService(tools = MyTool.class)
    interface MyService {
        Multi<String> chat(String msg);
    }

The reason is, that

https://github.com/quarkiverse/quarkus-langchain4j/blob/0cd6f69fc7092b67534e7d60ef2ec87df4d8f88d/core/runtime/src/main/java/io/quarkiverse/langchain4j/runtime/aiservice/AiServiceMethodImplementationSupport.java#L572

tries to execute the tool on the event loop.

I think this should be easily fixable by running it on Infrastructure.getDefaultWorkerPool().

I can contribute a fix, but implementing a test will not be possible without merging first #844.

This is not to be confused withe the request for supporting Uni as in #837.

[dastrobu]

There is a request on async tool execution in lang chain: https://github.com/langchain4j/langchain4j/issues/1183 but it doesn't seem to get implemented soon.

[geoand]

@langchain4j is receptive to PRs :). And I can help with the review as well

[langchain4j]

Hmm, I am not sure https://github.com/langchain4j/langchain4j/issues/1183 will help