vert-x3 / vertx-web

HTTP web applications for Vert.x
Apache License 2.0
1.11k stars 535 forks source link

Vertx web randomly returns 404 error for certain POST requests even though for other similar requests it returns 200 #2609

Closed sjmittal closed 6 months ago

sjmittal commented 6 months ago

Questions

Looks like vertx web randomly returns 404 error under high load.

Version

4.5.7

Context

I encountered an exception which looks suspicious while running vertx web on AWS ECS and connecting to a load balancer which receives very high traffic, somewhere like 1000 requetst per second.

Do you have a reproducer?

Here is how I have configured my router:

                Router router = Router.router(vertx);

                // add CORS and other handlers
                router.route().handler(LoggerHandler.create());
                router.route().handler(getCorsHandler());
                router.route().handler(ResponseContentTypeHandler.create());
                router.route().method(HttpMethod.POST).handler(BodyHandler.create());

                // handle get
                router.get("/StreamProducer").handler(new StreamProducerContextHandler());

                // handle get
                router
                    .get("/StreamProducer/analytics")
                    .produces("text/plain")
                    .handler(new AnalyticsHandler(...));

                // handle post
                router.post().produces("text/plain").handler(new PostHandler(...));

                // error handler
                router.route().failureHandler(getFailureHandler());

                // Serve the static resources
                router.route().handler(StaticHandler.create("webroot"));

                vertx.createHttpServer().requestHandler(router).listen(8080);

Steps to reproduce

  1. Run this as a docker container using AWS ECS.
  2. What I observe in logs for POST requests is something strange:
2024-05-15 12:18:36,962 [INFO ] [vert.x-eventloop-thread-1] [?:?] - 172.30.x.xxx - - [Wed, 15 May 2024 12:18:36 GMT] "POST /StreamProducer HTTP/1.1" 200 2 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.3 Safari/605.1.15"
2024-05-15 12:18:36,980 [WARN ] [vert.x-eventloop-thread-1] [?:?] - 172.30.x.xxx - - [Wed, 15 May 2024 12:18:36 GMT] "POST /StreamProducer HTTP/1.1" 404 53 "-" "Roku/DVP-12.5 (12.5.5.4174-50)"

You can clearly see that for same path it was able to process one request but for other it simply returns a 404 error. Any reason what could be the cause.

Extra

pendula95 commented 6 months ago

If I am not wrong, you have added produces("text/plain") to the POST route which means that requests that don't have header Accept with "text/plain" will not be routed to this route. First client requests probably contains this header and is able to route correctly but the second request does not contain this header which results in 404.

sjmittal commented 6 months ago

Thanks for pointing this out.