search

quarkusio / quarkus

Quarkus: Supersonic Subatomic Java.
https://quarkus.io
Apache License 2.0
13.88k stars 2.71k forks source link

2.4.1 causes strange HTTP failure in devmode #21282

Closed snowe2010 closed 2 years ago

snowe2010 commented 3 years ago

Describe the bug

When starting this example project, I receive repeating error messages, over and over again (without doing anything, literally just starting it up). The project was generated with quarkus.io, with the only modifications made to remove/add dependencies to duplicate the error. 

2021-11-05 12:29:05,388 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (vert.x-eventloop-thread-0) HTTP Request to /_lambda_/2018-06-01/runtime/init/error failed, error id: ba255bb1-447f-4e17-8e68-dacb2e2e42b6-1: java.util.concurrent.RejectedExecutionException
        at org.jboss.threads.RejectingExecutor.execute(RejectingExecutor.java:38)
        at org.jboss.threads.EnhancedQueueExecutor.rejectShutdown(EnhancedQueueExecutor.java:2076)

It appears to happen as soon as I enable the kogito-quarkus dependency.

Expected behavior

Dev mode should start without errors

Actual behavior

dev mode starts and throws a continuous stream of the above error.

How to Reproduce?

demo1.zip

  1. Start with mvn clean quarkus:dev
  2. Enable/disable kogito-quarkus dependency

Output of uname -a or ver

Darwin SR-MB-300245 21.1.0 Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE_X86_64 x86_64

Output of java -version

java version is specified using asdf in the .tool-versions file: java graalvm-21.0.0+java11

GraalVM version (if different from Java)

same as above. specified using asdf java graalvm-21.0.0+java11

Quarkus version or git rev

2.4.1

Build tool (ie. output of mvnw --version or gradlew --version)

also specified using asdf. maven 3.6.3

Additional information

logs-1636390731605.txt

geoand commented 3 years ago

This has nothing to do with kogito-quarkus , you can get a similar problem by just adding quarkus-vertx-web. When I was trying this, I consistently got:

ERROR [io.qua.run.Application] (Quarkus Main Thread) Port 8080 seems to be in use by another process. Quarkus may already be running or the port is used by another application.

This seems to be due to the fact that io.quarkus.amazon.lambda.runtime.MockEventServer is being started on port 8080 because of https://github.com/quarkusio/quarkus/blob/2.4.1.Final/extensions/amazon-lambda/common-deployment/src/main/java/io/quarkus/amazon/lambda/deployment/MockEventServerConfig.java#L15.

Now I have no idea how this is meant to work, so this is one for @patriot1burke

quarkus-bot[bot] commented 3 years ago

/cc @matejvasek, @patriot1burke

snowe2010 commented 3 years ago

@geoand great! I thought I had duplicated it with the vertx-web dependency as well, but then disabled it and it was still happening and had traced it to kogito-quarkus. Glad you were able to track down the real issue.

snowe2010 commented 3 years ago

In the meantime, is there anyway to 'unconfigure' that port?

patriot1burke commented 3 years ago

@geoand So, you cannot mix amazon-lambda plugin with quarkus-vertx-web. The Amazon Lambda dev service starts a MockEventServer under port 8080 so that you can invoke on your lambdas using http. See the docs for more info. You shouldn't be starting a web server within a lambda endpoint anyways!!!

Not sure what's going on with the kotlin-quarkus plugin. It may be trying to start its own web server?

patriot1burke commented 3 years ago

If you want to use HTTP libraries like resteasy, vertx-web, or servlet with lambda, you need the amazon-lambda-http or amazon-lambda-rest extensions. See docs

geoand commented 3 years ago

Thanks for the info @patriot1burke .

The quarkus-kotlin extension shouldn't be starting anything, but likely the kogito extension depends on Vertx

snowe2010 commented 3 years ago

@geoand So, you cannot mix amazon-lambda plugin with quarkus-vertx-web. The Amazon Lambda dev service starts a MockEventServer under port 8080 so that you can invoke on your lambdas using http. See the docs for more info. You shouldn't be starting a web server within a lambda endpoint anyways!!!

@patriot1burke well that would have been good to know. We've been writing our own endpoints and directing them at the lambda handler itself. Where is this documented? What is the exact endpoint to hit? just localhost:8080/? What's the data format?

@geoand so where should this get directed to?

snowe2010 commented 3 years ago

oh, I see it's in the guide. I thought those were all just tutorials, not that they would contain detailed info like that...

patriot1burke commented 3 years ago

@geoand So, you cannot mix amazon-lambda plugin with quarkus-vertx-web. The Amazon Lambda dev service starts a MockEventServer under port 8080 so that you can invoke on your lambdas using http. See the docs for more info. You shouldn't be starting a web server within a lambda endpoint anyways!!!

@patriot1burke well that would have been good to know. We've been writing our own endpoints and directing them at the lambda handler itself. Where is this documented? What is the exact endpoint to hit? just localhost:8080/? What's the data format?

https://quarkus.io/guides/amazon-lambda#live-coding-and-unitintegration-testing

Data format is whatever you want to POST. This is new stuff BTW introduced in 2.4.0.

snowe2010 commented 3 years ago

@patriot1burke ah, that explains it. I thought we had tried it last year and it didn't work. Ok, thanks!

evacchi commented 3 years ago

it looks like a it is a transitive dependency for quarkus-undertow, which is being addressed at https://github.com/kiegroup/kogito-runtimes/pull/1669

[INFO] |  +- io.quarkus:quarkus-undertow:jar:2.4.0.Final:compile
[INFO] |  |  +- io.quarkus:quarkus-vertx-http:jar:2.4.0.Final:compile
[INFO] |  |  |  +- io.quarkus:quarkus-security-runtime-spi:jar:2.4.0.Final:compile
[INFO] |  |  |  +- io.quarkus:quarkus-mutiny:jar:2.4.0.Final:compile
[INFO] |  |  |  |  +- io.quarkus:quarkus-smallrye-context-propagation:jar:2.4.0.Final:compile
[INFO] |  |  |  |  |  \- io.smallrye:smallrye-context-propagation:jar:1.2.2:compile
[INFO] |  |  |  |  |     +- io.smallrye:smallrye-context-propagation-api:jar:1.2.2:compile
[INFO] |  |  |  |  |     \- io.smallrye:smallrye-context-propagation-storage:jar:1.2.2:compile
[INFO] |  |  |  |  \- io.smallrye.reactive:mutiny-smallrye-context-propagation:jar:1.1.2:compile
[INFO] |  |  |  +- io.quarkus:quarkus-vertx-http-dev-console-runtime-spi:jar:2.4.0.Final:compile
[INFO] |  |  |  +- io.quarkus:quarkus-vertx:jar:2.4.0.Final:compile
[INFO] |  |  |  |  +- io.quarkus:quarkus-netty:jar:2.4.0.Final:compile
[INFO] |  |  |  |  +- io.netty:netty-codec-haproxy:jar:4.1.68.Final:compile
[INFO] |  |  |  |  +- io.quarkus:quarkus-jackson:jar:2.4.0.Final:compile
[INFO] |  |  |  |  |  +- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:2.12.5:compile
[INFO] |  |  |  |  |  \- com.fasterxml.jackson.module:jackson-module-parameter-names:jar:2.12.5:compile
[INFO] |  |  |  |  +- io.smallrye.reactive:smallrye-mutiny-vertx-core:jar:2.14.1:compile
[INFO] |  |  |  |  |  +- io.smallrye.reactive:smallrye-mutiny-vertx-runtime:jar:2.14.1:compile
[INFO] |  |  |  |  |  \- io.smallrye.reactive:vertx-mutiny-generator:jar:2.14.1:compile
[INFO] |  |  |  |  |     \- io.vertx:vertx-codegen:jar:4.1.5:compile
[INFO] |  |  |  |  \- io.smallrye:smallrye-fault-tolerance-vertx:jar:5.2.1:compile
[INFO] |  |  |  +- io.smallrye.reactive:smallrye-mutiny-vertx-web:jar:2.14.1:compile
[INFO] |  |  |  |  +- io.smallrye.reactive:smallrye-mutiny-vertx-web-common:jar:2.14.1:compile
[INFO] |  |  |  |  +- io.smallrye.reactive:smallrye-mutiny-vertx-auth-common:jar:2.14.1:compile
[INFO] |  |  |  |  \- io.smallrye.reactive:smallrye-mutiny-vertx-bridge-common:jar:2.14.1:compile
[INFO] |  |  |  \- io.vertx:vertx-web:jar:4.1.5:compile
[INFO] |  |  |     +- io.vertx:vertx-web-common:jar:4.1.5:compile
[INFO] |  |  |     +- io.vertx:vertx-auth-common:jar:4.1.5:compile
[INFO] |  |  |     \- io.vertx:vertx-bridge-common:jar:4.1.5:compile

I think you can exclude quarkus-undertow from kogito-quarkus in the meantime

snowe2010 commented 3 years ago

@patriot1burke Creating an app on quarkus.io with just the kotlin, lambda, and rest-client options results in an unrunnable build due to the 8080 thing. It automatically adds all the rest assured/rest easy stuff. I think that's why I encountered this issue.

snowe2010 commented 3 years ago

@patriot1burke it also appears that kogito depends on using vertx regardless, so I'm not sure how I'm supposed to remove the dependency here. Following Edoardo's suggestion just results in more and more errors, the more dependencies I exclude. A config option to disable or change this port would probably resolve the problem until it can get figured out a different way.

snowe2010 commented 3 years ago

turns out you can do so with 

quarkus.lambda.mock-event-server.dev-port=8082
quarkus.lambda.mock-event-server.test-port=8083
gsmet commented 2 years ago

@patriot1burke maybe we should have a proper message error when Vert.x/RESTEasy is around and using the same port?

The other option would be to document the error message and how to solve it.

patriot1burke commented 2 years ago

@gsmet The problem is that people are using extensions that start up an HTTP server within a vanilla AWS Lambda. Some of these extensions like Kogito require an HTTP server (at least I think it requires HTTP) and should not be used within a vanilla AWS Lambda. Other extensions like quarkus-oidc-client are incorrectly pulling in the vertx-http-deployment dependency and thus starting an HTTP server.

I can have Vert.x HTTP publish either an "vertx-http" feature or a marker build item. Check within lambda processor for this and give a warning or abort with message.

snowe2010 commented 2 years ago

@patriot1burke so how are you 'supposed' to be using kogito then? It's built to run on AWS lambda is it not? And I'm glad to see that I wasn't crazy when I was seeing the same issues with OIDC. Is this all a consequence of the devmode http handler so that you can hit lambdas locally?

evacchi commented 2 years ago

Kogito was requiring undertow to serve static resources, this should be no longer the case for 1.15.0. We also automatically generate rest endpoints, which may conflict with AWS. You should be able to opt-out from automated rest generation setting to your properties:

kogito.generate.rest=false

then you should be able to try the new experimental public API https://blog.kie.org/2021/11/the-road-towards-a-kogito-public-api.html

I would also advise to check with Kogito 1.16, which should be release together with Quarkus 2.7.0, for all latest improvements.

patriot1burke commented 2 years ago

@evacchi Yeah, Kogito was pulling in quarkus-vertx-http and thus starting an HTTP endpoint.

patriot1burke commented 2 years ago

this all a consequence of the devmode http handler so that you can hit lambdas locally?

@snowe2010 Its more that this devmode change uncovered the fact that there are a few extensions that SHOULD NOT be booting up an http server and that were implemented incorrectly.

evacchi commented 2 years ago

so I think this has been addressed with https://github.com/kiegroup/kogito-runtimes/pull/1757

patriot1burke commented 2 years ago

@evacchi I don't know... grep for 'quarkus-vertx-http' or quarkus-vertx-http-deployment in your poms. Last time I looked it was still pulling this in.

evacchi commented 2 years ago

currently this is all I am finding:

./quarkus/extensions/kogito-quarkus-extension-common/kogito-quarkus-common-deployment/pom.xml:            <artifactId>quarkus-vertx-http-deployment-spi</artifactId>

which should be safe

geoand commented 2 years ago

which should be safe

👍🏼

Let's close the issue. If it's still a problem when Quarkus 2.7 and the corresponding Kogito 1.16 releases are out, we can reopem