search

quarkiverse / quarkus-langchain4j

Quarkus Langchain4j extension
https://docs.quarkiverse.io/quarkus-langchain4j/dev/index.html
Apache License 2.0
146 stars 88 forks source link

problem running email-a-poem sample #808

Closed codefromthecrypt closed 3 months ago

codefromthecrypt commented 3 months ago

I'm trying to use the email-a-poem sample with ollama. I tried to switch the dep and even add a property for the unused openapi key. However, if I do mvn quarkus:run from the email-a-pom dir, it crashes for a similar reason. Any idea what I'm doing wrong? My goal is to test otel and this seems like the only example integrated with it.

$ mvn quarkus:run
[INFO] Scanning for projects...
[INFO] 
[INFO] -----< io.quarkiverse.langchain4j:quarkus-langchain4j-sample-poem >-----
[INFO] Building Quarkus LangChain4j - Sample - Email me a poem 1.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- quarkus:3.12.1:run (default-cli) @ quarkus-langchain4j-sample-poem ---
[INFO] [io.quarkiverse.langchain4j.deployment.devservice.DevServicesOllamaProcessor] Preloading model llama3.1
[INFO] Found Docker environment with Testcontainers Host with tc.host=tcp://127.0.0.1:49832
[INFO] Docker host IP address is 127.0.0.1
[INFO] Connected to docker: 
  Server Version: 82+testcontainerscloud (via Testcontainers Desktop 1.15.0)
  API Version: 1.46
  Operating System: Ubuntu 22.04.4 LTS
  Total Memory: 31548 MB
[INFO] Image pull policy will be performed by: DefaultPullPolicy()
[INFO] Image name substitution will be performed by: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor')
[INFO] Checking the system...
[INFO] ✔︎ Docker server version should be at least 1.6.0
[INFO] Creating container for image: axllent/mailpit:latest
[INFO] Creating container for image: testcontainers/ryuk:0.7.0
[INFO] Container testcontainers/ryuk:0.7.0 is starting: 25f4788787434ec3ae6fafbc32a2041ec37ee72c31669111d93fbbdbc74d7eac
[INFO] Container testcontainers/ryuk:0.7.0 started in PT1.395458S
[INFO] Container axllent/mailpit:latest is starting: 9c0a36395da9393c3aea7724fdd5ceddbb89b28b8bdf7d202458c4d384811393
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 05:54:53" level=debug msg="[db] using temporary database: /tmp/mailpit-1723096493841728457.db"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 05:54:53" level=debug msg="[db] applied schema: 1.0.0.sql"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 05:54:53" level=debug msg="[db] applied schema: 1.1.0.sql"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 05:54:53" level=debug msg="[db] applied schema: 1.2.0.sql"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 05:54:53" level=debug msg="[db] applied schema: 1.3.0.sql"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 05:54:53" level=debug msg="[db] applied schema: 1.4.0.sql"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 05:54:53" level=debug msg="[db] applied schema: 1.5.0.sql"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 05:54:53" level=info msg="[smtpd] starting on [::]:1025 (no encryption)"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 05:54:53" level=info msg="[http] starting on [::]:8025"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 05:54:53" level=info msg="[http] accessible via http://localhost:8025/q/mailpit/"
[INFO] /keen_shtern: Waiting for 60 seconds for URL: http://127.0.0.1:32782/q/mailpit (where port 32782 maps to container port 8025)
[INFO] Container axllent/mailpit:latest started in PT4.129683S
[INFO] [io.quarkiverse.mailpit.deployment.MailpitProcessor] Dev Services for Mailpit started.
Executing "/Users/adriancole/Library/Java/JavaVirtualMachines/temurin-17.0.11/Contents/Home/bin/java -Dquarkus.platform.version=3.12.1 -Dquarkus.application.version=1.0-SNAPSHOT -Dquarkus.platform.artifact-id=quarkus-bom -Dquarkus.platform.group-id=io.quarkus -Dquarkus.application.name=quarkus-langchain4j-sample-poem -Dmailpit.http.server=http://127.0.0.1:32782 -Dquarkus.mailer.host=127.0.0.1 -Dquarkus.mailer.mock=false -Dmailpit.http.port=32782 -DMP_WEBROOT=/q/mailpit -DMP_VERBOSE=true -Dquarkus.langchain4j.ollama.base-url=http://localhost:11434 -Dmailpit.smtp.port=32781 -Dquarkus.mailer.port=32781 -Dmailpit.http.host=127.0.0.1 -DMP_MAX_MESSAGES=500 -jar /Users/adriancole/oss/quarkus-langchain4j/samples/email-a-poem/target/quarkus-app/quarkus-run.jar"
__  ____  __  _____   ___  __ ____  ______ 
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2024-08-08 13:55:00,519 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.otel.exporter.otlp.traces.protocol" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2024-08-08 13:55:00,519 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.langchain4j.ollama.base-url" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2024-08-08 13:55:00,762 ERROR [io.qua.run.Application] (main) Failed to start application (with profile [prod]): java.lang.RuntimeException: Failed to start quarkus
        at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
        at io.quarkus.runtime.Application.start(Application.java:101)
        at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:111)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:44)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:124)
        at io.quarkus.runner.GeneratedMain.main(Unknown Source)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doRun(QuarkusEntryPoint.java:62)
        at io.quarkus.bootstrap.runner.QuarkusEntryPoint.main(QuarkusEntryPoint.java:33)
Caused by: io.smallrye.config.ConfigValidationException: Configuration validation failed:
        SRCFG00014: The config property quarkus.langchain4j.openai.api-key is required but it could not be found in any config source
        at io.quarkiverse.langchain4j.openai.runtime.OpenAiRecorder.chatModel(OpenAiRecorder.java:58)
        at io.quarkus.deployment.steps.OpenAiProcessor$generateBeans801072037.deploy_0(Unknown Source)
        at io.quarkus.deployment.steps.OpenAiProcessor$generateBeans801072037.deploy(Unknown Source)
        ... 13 more
--- a/samples/email-a-poem/pom.xml
+++ b/samples/email-a-poem/pom.xml
@@ -40,7 +40,7 @@
         </dependency>
         <dependency>
             <groupId>io.quarkiverse.langchain4j</groupId>
-            <artifactId>quarkus-langchain4j-openai</artifactId>
+            <artifactId>quarkus-langchain4j-ollama</artifactId>
             <version>${quarkus-langchain4j.version}</version>
         </dependency>
         <dependency>
diff --git a/samples/email-a-poem/src/main/resources/application.properties b/samples/email-a-poem/src/main/resources/application.properties
index 36ae48b5..d02e4525 100644
--- a/samples/email-a-poem/src/main/resources/application.properties
+++ b/samples/email-a-poem/src/main/resources/application.properties
@@ -1,7 +1,7 @@
 quarkus.langchain4j.timeout=60s
 quarkus.langchain4j.log-requests=true
 quarkus.langchain4j.log-responses=true
-
+quarkus.langchain4j.openai.api-key=unused
 quarkus.langchain4j.openai.chat-model.model-name=gpt-4o
geoand commented 3 months ago

Hi,

I just tried exactly that and it worked without issue. 

diff --git a/samples/email-a-poem/pom.xml b/samples/email-a-poem/pom.xml
index 8b04e35c..56ddae3c 100644
--- a/samples/email-a-poem/pom.xml
+++ b/samples/email-a-poem/pom.xml
@@ -40,7 +40,7 @@
         </dependency>
         <dependency>
             <groupId>io.quarkiverse.langchain4j</groupId>
-            <artifactId>quarkus-langchain4j-openai</artifactId>
+            <artifactId>quarkus-langchain4j-ollama</artifactId>
             <version>${quarkus-langchain4j.version}</version>
         </dependency>
         <dependency>
diff --git a/samples/email-a-poem/src/main/resources/application.properties b/samples/email-a-poem/src/main/resources/application.properties
index 36ae48b5..368dc54d 100644
--- a/samples/email-a-poem/src/main/resources/application.properties
+++ b/samples/email-a-poem/src/main/resources/application.properties
@@ -2,7 +2,7 @@ quarkus.langchain4j.timeout=60s
 quarkus.langchain4j.log-requests=true
 quarkus.langchain4j.log-responses=true

-quarkus.langchain4j.openai.chat-model.model-name=gpt-4o
+#quarkus.langchain4j.openai.chat-model.model-name=gpt-4o

 # mailer config
 quarkus.mailer.from=demoer@langchain4j.ai

Keep in mind that if you want to test the OTel stuff, you need to start the sample with mvn quarkus:dev -Dobservability (which brings in the observability related Quarkus extensions).

geoand commented 3 months ago

@codefromthecrypt do you mind zipping the sample and adding it here if it continues to missbehave?

codefromthecrypt commented 3 months ago

So, I'm getting closer, and bear in mind I do not want to run the lgtm stack, that's why I was using run. I want to verify the traces with my own collector. When I run from top-level and then do a full build followed by quarkus:run, the server starts, and the form presents on :8080, but it has "Failed! Have you started the mock mail container?" in it. any ideas?

$ ./mvnw clean install -Dobservability -DskipTests
--snip--
$ ./mvnw quarkus:run -Dobservability -pl ./samples/email-a-poem 
[INFO] Scanning for projects...
[INFO] 
[INFO] -----< io.quarkiverse.langchain4j:quarkus-langchain4j-sample-poem >-----
[INFO] Building Quarkus LangChain4j - Sample - Email me a poem 1.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- quarkus:3.12.1:run (default-cli) @ quarkus-langchain4j-sample-poem ---
[INFO] [io.quarkiverse.langchain4j.deployment.devservice.DevServicesOllamaProcessor] Preloading model llama3.1
[INFO] Found Docker environment with Testcontainers Host with tc.host=tcp://127.0.0.1:49832
[INFO] Docker host IP address is 127.0.0.1
[INFO] Connected to docker: 
  Server Version: 82+testcontainerscloud (via Testcontainers Desktop 1.15.0)
  API Version: 1.46
  Operating System: Ubuntu 22.04.4 LTS
  Total Memory: 31548 MB
[INFO] Image pull policy will be performed by: DefaultPullPolicy()
[INFO] Image name substitution will be performed by: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor')
[INFO] Checking the system...
[INFO] ✔︎ Docker server version should be at least 1.6.0
[INFO] Creating container for image: axllent/mailpit:latest
[INFO] Creating container for image: testcontainers/ryuk:0.7.0
[INFO] Container testcontainers/ryuk:0.7.0 is starting: 03fd9abda9114536091b14ff20e0dfbbb6dad5f15163ebeba167a1ba21393757
[INFO] Container testcontainers/ryuk:0.7.0 started in PT1.380251S
[INFO] Container axllent/mailpit:latest is starting: b5230758f03b7601f9c774290de8e3ad4c64f6dd189dd5a594c76b760c3ac88e
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 07:55:47" level=debug msg="[db] using temporary database: /tmp/mailpit-1723103747764650189.db"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 07:55:47" level=debug msg="[db] applied schema: 1.0.0.sql"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 07:55:47" level=debug msg="[db] applied schema: 1.1.0.sql"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 07:55:47" level=debug msg="[db] applied schema: 1.2.0.sql"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 07:55:47" level=debug msg="[db] applied schema: 1.3.0.sql"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 07:55:47" level=debug msg="[db] applied schema: 1.4.0.sql"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 07:55:47" level=debug msg="[db] applied schema: 1.5.0.sql"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 07:55:47" level=info msg="[smtpd] starting on [::]:1025 (no encryption)"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 07:55:47" level=info msg="[http] starting on [::]:8025"
[INFO] [io.quarkiverse.mailpit.deployment.MailpitContainer] [mailpit] STDOUT: time="2024/08/08 07:55:47" level=info msg="[http] accessible via http://localhost:8025/q/mailpit/"
[INFO] /agitated_margulis: Waiting for 60 seconds for URL: http://127.0.0.1:32817/q/mailpit (where port 32817 maps to container port 8025)
[INFO] Container axllent/mailpit:latest started in PT4.265154S
[INFO] [io.quarkiverse.mailpit.deployment.MailpitProcessor] Dev Services for Mailpit started.
Executing "/Users/adriancole/Library/Java/JavaVirtualMachines/temurin-17.0.11/Contents/Home/bin/java -Dquarkus.platform.version=3.12.1 -Dquarkus.application.version=1.0-SNAPSHOT -Dquarkus.platform.artifact-id=quarkus-bom -Dquarkus.platform.group-id=io.quarkus -Dquarkus.application.name=quarkus-langchain4j-sample-poem -Dmailpit.http.server=http://127.0.0.1:32817 -Dquarkus.mailer.host=127.0.0.1 -Dquarkus.mailer.mock=false -Dmailpit.http.port=32817 -DMP_WEBROOT=/q/mailpit -DMP_VERBOSE=true -Dquarkus.langchain4j.ollama.base-url=http://localhost:11434 -Dmailpit.smtp.port=32816 -Dquarkus.mailer.port=32816 -Dmailpit.http.host=127.0.0.1 -DMP_MAX_MESSAGES=500 -jar /Users/adriancole/oss/quarkus-langchain4j/samples/email-a-poem/target/quarkus-app/quarkus-run.jar"
__  ____  __  _____   ___  __ ____  ______ 
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2024-08-08 15:55:55,034 INFO  [io.quarkus] (main) quarkus-langchain4j-sample-poem 1.0-SNAPSHOT on JVM (powered by Quarkus 3.12.1) started in 5.602s. Listening on: http://0.0.0.0:8080
2024-08-08 15:55:55,037 INFO  [io.quarkus] (main) Profile prod activated. 
2024-08-08 15:55:55,037 INFO  [io.quarkus] (main) Installed features: [cdi, langchain4j, langchain4j-ollama, mailer, micrometer, opentelemetry, qute, rest, rest-client, rest-client-jackson, rest-jackson, smallrye-context-propagation, vertx]
codefromthecrypt commented 3 months ago

ps in above the collector works, just the app is busted due to the mock mailer somehow

Screenshot 2024-08-08 at 3 56 48 PM Screenshot 2024-08-08 at 3 56 21 PM
geoand commented 3 months ago

So, I'm getting closer, and bear in mind I do not want to run the lgtm stack, that's why I was using run

Oh, nice!

I'll look at the mailer issue soon and get back to you, there might be some kind of bug with quarkus:run - it should work with quarkus:dev

geoand commented 3 months ago

Oddly enough the error is happening in the browser (trying to figure it out now). If you do:

curl http://localhost:8080/email-me-a-poem

things should work

geoand commented 3 months ago

Oh, now I see why the browser fails...

When the application is run with quarkus:run, it listens on all network hosts, so Quarkus prints:

Listening on: http://0.0.0.0:8080

When you try to use that in the browser, you get a CORS error when clicking on the button.

If you instead use http://localhost:8080 everything works.

Let me see if non-existent JS skills can address the problem.

geoand commented 3 months ago

https://github.com/quarkiverse/quarkus-langchain4j/commit/0f7746a0a9cc21108e65caad615b0175d251aaec fixes the issue :)

iocanel commented 3 months ago

@codefromthecrypt It's been a while!

Your js console might reveal more information on what's going on, as the message implies that the HTTP request towards http://localhost:8080/email-me-a-poem barfed.

geoand commented 3 months ago

@iocanel problem fixed :)

codefromthecrypt commented 3 months ago

@iocanel I just can't quit you 🤠

codefromthecrypt commented 3 months ago

cool, it works. I think we need a clean build with the profile enabled first. If there are less steps we can pare it down

Screenshot 2024-08-08 at 4 48 13 PM

This is https://github.com/ymtdzzz/otel-tui and I am their ⭐ promotor, btw.

not sure I needed the endpoint property but hey

diff --git a/samples/email-a-poem/pom.xml b/samples/email-a-poem/pom.xml
index 8b04e35c..56ddae3c 100644
--- a/samples/email-a-poem/pom.xml
+++ b/samples/email-a-poem/pom.xml
@@ -40,7 +40,7 @@
         </dependency>
         <dependency>
             <groupId>io.quarkiverse.langchain4j</groupId>
-            <artifactId>quarkus-langchain4j-openai</artifactId>
+            <artifactId>quarkus-langchain4j-ollama</artifactId>
             <version>${quarkus-langchain4j.version}</version>
         </dependency>
         <dependency>
diff --git a/samples/email-a-poem/src/main/resources/application.properties b/samples/email-a-poem/src/main/resources/application.properties
index 36ae48b5..602297b3 100644
--- a/samples/email-a-poem/src/main/resources/application.properties
+++ b/samples/email-a-poem/src/main/resources/application.properties
@@ -2,12 +2,14 @@ quarkus.langchain4j.timeout=60s
 quarkus.langchain4j.log-requests=true
 quarkus.langchain4j.log-responses=true

-quarkus.langchain4j.openai.chat-model.model-name=gpt-4o
+#quarkus.langchain4j.openai.chat-model.model-name=gpt-4o

 # mailer config
 quarkus.mailer.from=demoer@langchain4j.ai

 # observability config
 quarkus.otel.exporter.otlp.traces.protocol=http/protobuf
+quarkus.otel.exporter.otlp.endpoint=http://localhost:4318
+
 %test.quarkus.otel.exporter.otlp.traces.endpoint=http://${quarkus.otel-collector.url}
 %dev.quarkus.otel.exporter.otlp.traces.endpoint=http://${quarkus.otel-collector.url}
geoand commented 3 months ago

This is https://github.com/ymtdzzz/otel-tui and I am their ⭐ promotor, btw.

Really really cool!

Yeah, I'm thinking about making the observability stuff enabled by default - the reason we haven't done it so far is because it results in the downloading of a large docker image, but I think we can live with that.

geoand commented 3 months ago

@brunobat WDYT about making https://github.com/ymtdzzz/otel-tui available as a dev-service?

codefromthecrypt commented 3 months ago

if it helps, this is some notes I have until I make a collection of demos somewhere. Also very soon otel-tui will work with zipkin, too (not sure that matters here)

services:
  otel-collector:
    extends:
      file: docker-compose.yml
      service: otel-collector
    image: ymtdzzz/otel-tui:latest
    stdin_open: true
    tty: true
    command: []
    volumes: []

Run otel-tui instead of collector

If you want to view a text terminal UI instead of the collector, you can use otel-tui like this:

Use docker-compose to run ollama and an otel-tui in the background:

docker-compose -f docker-compose.yml -f docker-compose-tui.yml up -d

Then, attach your terminal to the terminal UI like this:

docker-compose -f docker-compose.yml -f docker-compose-tui.yml attach otel-collector

To detach from otel-tui, use Ctrl-p Ctrl-q.

Finally, to tear down otel-tui and ollama, execute this:

docker-compose -f docker-compose.yml -f docker-compose-tui.yml down
geoand commented 3 months ago

🙏🏼

brunobat commented 2 months ago

@brunobat WDYT about making https://github.com/ymtdzzz/otel-tui available as a dev-service?

This is cool... What do you think @ales?