search

quarkiverse / quarkus-langchain4j

Quarkus Langchain4j extension
https://docs.quarkiverse.io/quarkus-langchain4j/dev/index.html
Apache License 2.0
146 stars 88 forks source link

EasyRag when multiple providers are present erroring when running tests #786

Closed edeandrea closed 3 months ago

edeandrea commented 3 months ago

I'm working on a workshop that uses EasyRag. It's a pretty simple project. The project has multiple provider extensions present (openai & ollama).

https://github.com/rh-rad-ai-roadshow/parasol-insurance/tree/private-docs-rag-solution/app

My application.properties has (among other things)

quarkus.langchain4j.parasol-chat.chat-model.provider=ollama # There are multiple chat models
quarkus.langchain4j.embedding-model.provider=ollama # There is only a single embedding model

When I run in dev mode things seem to be fine, but when I run ./mvnw verify it blows up:

[ERROR] org.parasol.resources.ClaimResourceTests.getAllNoneFound -- Time elapsed: 0.002 s <<< ERROR!
java.lang.RuntimeException: 
java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
        [error]: Build step io.quarkiverse.langchain4j.deployment.BeansProcessor#handleProviders threw an exception: io.quarkus.runtime.configuration.ConfigurationException: A EmbeddingModel bean was requested, but since there are multiple available providers, the 'quarkus.langchain4j.embedding-model.provider' needs to be set to one of the available options (openai,ollama).
        at io.quarkiverse.langchain4j.deployment.BeansProcessor.selectEmbeddingModelProvider(BeansProcessor.java:333)
        at io.quarkiverse.langchain4j.deployment.BeansProcessor.handleProviders(BeansProcessor.java:175)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
        at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
        at java.base/java.lang.Thread.run(Thread.java:1583)
        at org.jboss.threads.JBossThread.run(JBossThread.java:483)

        at io.quarkus.test.junit.QuarkusTestExtension.throwBootFailureException(QuarkusTestExtension.java:642)
        at io.quarkus.test.junit.QuarkusTestExtension.interceptTestClassConstructor(QuarkusTestExtension.java:726)
        at java.base/java.util.Optional.orElseGet(Optional.java:364)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
Caused by: java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
        [error]: Build step io.quarkiverse.langchain4j.deployment.BeansProcessor#handleProviders threw an exception: io.quarkus.runtime.configuration.ConfigurationException: A EmbeddingModel bean was requested, but since there are multiple available providers, the 'quarkus.langchain4j.embedding-model.provider' needs to be set to one of the available options (openai,ollama).
        at io.quarkiverse.langchain4j.deployment.BeansProcessor.selectEmbeddingModelProvider(BeansProcessor.java:333)
        at io.quarkiverse.langchain4j.deployment.BeansProcessor.handleProviders(BeansProcessor.java:175)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
        at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
        at java.base/java.lang.Thread.run(Thread.java:1583)
        at org.jboss.threads.JBossThread.run(JBossThread.java:483)

        at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:331)
        at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:252)
        at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:58)
        at io.quarkus.test.junit.QuarkusTestExtension.doJavaStart(QuarkusTestExtension.java:223)
        at io.quarkus.test.junit.QuarkusTestExtension.ensureStarted(QuarkusTestExtension.java:609)
        at io.quarkus.test.junit.QuarkusTestExtension.beforeAll(QuarkusTestExtension.java:659)
        ... 1 more
Caused by: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
        [error]: Build step io.quarkiverse.langchain4j.deployment.BeansProcessor#handleProviders threw an exception: io.quarkus.runtime.configuration.ConfigurationException: A EmbeddingModel bean was requested, but since there are multiple available providers, the 'quarkus.langchain4j.embedding-model.provider' needs to be set to one of the available options (openai,ollama).
        at io.quarkiverse.langchain4j.deployment.BeansProcessor.selectEmbeddingModelProvider(BeansProcessor.java:333)
        at io.quarkiverse.langchain4j.deployment.BeansProcessor.handleProviders(BeansProcessor.java:175)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
        at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
        at java.base/java.lang.Thread.run(Thread.java:1583)
        at org.jboss.threads.JBossThread.run(JBossThread.java:483)

        at io.quarkus.builder.Execution.run(Execution.java:123)
        at io.quarkus.builder.BuildExecutionBuilder.execute(BuildExecutionBuilder.java:79)
        at io.quarkus.deployment.QuarkusAugmentor.run(QuarkusAugmentor.java:160)
        at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:327)
        ... 6 more
Caused by: io.quarkus.runtime.configuration.ConfigurationException: A EmbeddingModel bean was requested, but since there are multiple available providers, the 'quarkus.langchain4j.embedding-model.provider' needs to be set to one of the available options (openai,ollama).
        at io.quarkiverse.langchain4j.deployment.BeansProcessor.selectEmbeddingModelProvider(BeansProcessor.java:333)
        at io.quarkiverse.langchain4j.deployment.BeansProcessor.handleProviders(BeansProcessor.java:175)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
        at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
        at java.base/java.lang.Thread.run(Thread.java:1583)
        at org.jboss.threads.JBossThread.run(JBossThread.java:483)
geoand commented 3 months ago

Does the same thing happen if playwright is not used?

weand commented 3 months ago

I had similar issues, where quarkus was complaining about A EmbeddingModel bean was requested, but since there are multiple available providers, the 'quarkus.langchain4j.embedding-model.provider' while a had the configuration set properly.

I tried to use an in process embedding model in the chatbot-easy-rag example with these dependencies

        <dependency>
            <groupId>io.quarkiverse.langchain4j</groupId>
            <artifactId>quarkus-langchain4j-openai</artifactId>
            <version>0.16.4</version>
        </dependency>
        <dependency>
            <groupId>dev.langchain4j</groupId>
            <artifactId>langchain4j-embeddings-bge-small-en-q</artifactId>
            <version>0.32.0</version>
        </dependency>

added this to the application.properties 

quarkus.langchain4j.embedding-model.provider=dev.langchain4j.model.embedding.BgeSmallEnQuantizedEmbeddingModel

which results in

[ERROR]         [error]: Build step io.quarkiverse.langchain4j.deployment.BeansProcessor#handleProviders threw an exception: io.quarkus.runtime.configuration.ConfigurationException: A EmbeddingModel bean
 was requested, but since there are multiple available providers, the 'quarkus.langchain4j.embedding-model.provider' needs to be set to one of the available options (openai,dev.langchain4j.model.embeddin
g.BgeSmallEnQuantizedEmbeddingModel).
[ERROR]         at io.quarkiverse.langchain4j.deployment.BeansProcessor.selectEmbeddingModelProvider(BeansProcessor.java:333)
[ERROR]         at io.quarkiverse.langchain4j.deployment.BeansProcessor.handleProviders(BeansProcessor.java:175)

I found a workaround by adding a bean like this, which causes quarkus to read the configuration properly.

@Singleton
@Startup
public class Ingestion {

    public Ingestion(EmbeddingModel embedding) {
        Log.info(embedding.getClass());
    }

}

Might help here as well. Seems like a bug.

geoand commented 3 months ago

@weand I just tried exactly what you described from the current status of the repo, and everything worked as expected without me hacing to create my own bean

edeandrea commented 3 months ago

Does the same thing happen if playwright is not used?

Even if i take out Playwright I still see this issue.

weand commented 3 months ago

@weand I just tried exactly what you described from the current status of the repo, and everything worked as expected without me hacing to create my own bean

with quarkus-langchain4j 0.16.4 its giving me the error. And yes, using 0.17.0.CR1 seems to help as well, thanks. So updating version might help the thread author here too. It's not clear to me, if it was an intentional fix.

edeandrea commented 3 months ago

Updating to 0.17.0.CR1 does not fix the issue for me.

edeandrea commented 3 months ago

Even just doing ./mvnw clean package -DskipTests blows up with both 0.16.4 and 0.17.0.CR1.

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  6.126 s
[INFO] Finished at: 2024-07-29T10:10:14-04:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.quarkus.platform:quarkus-maven-plugin:3.12.3:build (default) on project insurance-app: Failed to build quarkus application: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[ERROR]         [error]: Build step io.quarkiverse.langchain4j.deployment.BeansProcessor#handleProviders threw an exception: io.quarkus.runtime.configuration.ConfigurationException: A EmbeddingModel bean was requested, but since there are multiple available providers, the 'quarkus.langchain4j.embedding-model.provider' needs to be set to one of the available options (ollama,openai).
[ERROR]         at io.quarkiverse.langchain4j.deployment.BeansProcessor.selectEmbeddingModelProvider(BeansProcessor.java:333)
[ERROR]         at io.quarkiverse.langchain4j.deployment.BeansProcessor.handleProviders(BeansProcessor.java:175)
[ERROR]         at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
[ERROR]         at java.base/java.lang.reflect.Method.invoke(Method.java:580)
[ERROR]         at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
[ERROR]         at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
[ERROR]         at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
[ERROR]         at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
[ERROR]         at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
[ERROR]         at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
[ERROR]         at java.base/java.lang.Thread.run(Thread.java:1583)
[ERROR]         at org.jboss.threads.JBossThread.run(JBossThread.java:483)
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
geoand commented 3 months ago

If you have a branch that doesn't include Playwright , I can take a look tomorrow

weand commented 3 months ago

I must correct myself. 0.17.0.CR1 is also not working when using my local model. I didn't checked that for my earlier comment. So adding something like this breaks it again quarkus.langchain4j.openai.base-url=http://localhost:8000/v1 -> A EmbeddingModel bean was requested, but since there are multiple available providers, the 'quarkus.langchain4j.embedding-model.provider' needs to be set to one of the available options

@edeandrea I ran mvn clean package -DskipTests successfully on your sample branch after adding my first suggested workaround...

package org.parasol.ai;

import dev.langchain4j.model.embedding.EmbeddingModel;
import io.quarkus.logging.Log;
import io.quarkus.runtime.Startup;
import jakarta.inject.Singleton;

@Singleton
@Startup
public class Noop {

    public Noop(EmbeddingModel embedding) {
        Log.info(embedding.getClass());
    }

}
edeandrea commented 3 months ago

@weand Your solution does indeed work for me (with and without Playwright involved).

After adding that dummy Noop class into my source tree everything works as expected.

@geoand ^^^

geoand commented 3 months ago

That comes as no surprise really

edeandrea commented 3 months ago

So do you still need a branch that doesnt include Playwright?

geoand commented 3 months ago

It would be nice

edeandrea commented 3 months ago

@geoand

https://github.com/edeandrea/parasol-insurance/tree/quarkus-langchain4j-issues/app

Run ./mvnw clean package -DskipTests and see

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  24.321 s
[INFO] Finished at: 2024-07-29T11:26:22-04:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.quarkus.platform:quarkus-maven-plugin:3.12.3:build (default) on project insurance-app: Failed to build quarkus application: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[ERROR]         [error]: Build step io.quarkiverse.langchain4j.deployment.BeansProcessor#handleProviders threw an exception: io.quarkus.runtime.configuration.ConfigurationException: A EmbeddingModel bean was requested, but since there are multiple available providers, the 'quarkus.langchain4j.embedding-model.provider' needs to be set to one of the available options (ollama,openai).
[ERROR]         at io.quarkiverse.langchain4j.deployment.BeansProcessor.selectEmbeddingModelProvider(BeansProcessor.java:333)
[ERROR]         at io.quarkiverse.langchain4j.deployment.BeansProcessor.handleProviders(BeansProcessor.java:175)
[ERROR]         at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
[ERROR]         at java.base/java.lang.reflect.Method.invoke(Method.java:580)
[ERROR]         at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
[ERROR]         at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
[ERROR]         at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
[ERROR]         at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
[ERROR]         at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
[ERROR]         at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
[ERROR]         at java.base/java.lang.Thread.run(Thread.java:1583)
[ERROR]         at org.jboss.threads.JBossThread.run(JBossThread.java:483)
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
geoand commented 3 months ago

🙏

geoand commented 3 months ago

@jmartisk the difference between dev and test mode here happens because of EmbeddingStoreJsonRPCService. I would argue that this difference is very surprising for users and almost certainly not what we want

jmartisk commented 3 months ago

Yeah that class injects EmbeddingStore and EmbeddingModel directly and thus declares a hard dependency, I think it should wrap these injection points into an Instance and that should fix it. I'll give it a try