search

Knotx / knotx

Knot.x is a highly-efficient and scalable integration framework designed to build backend APIs
https://knotx.io
Apache License 2.0
126 stars 26 forks source link

Incorrect handling of ESI calls initiated after POST request #479

Open jwadolowski opened 5 years ago

jwadolowski commented 5 years ago

Bug description

knot.x version: 1.4.0

Email signup is one of the features on my website. User initiates POST request which bypasses knot.x completely. The response contains ESI markup, so CDN triggers sub-requests, which go via knot.x instance. knot.x returns 500 for each subrequest call.

This situation doesn't happen when the entire flow starts with GET request.

Here's a diagram with end-to-end HTTP flow:

kx_req_flow

Steps to reproduce

  1. Send a POST request with Content-Type: application/x-www-form-urlencoded header
  2. Make sure response contains ESI markup
  3. ESI sub-requests should go via knot.x (they inherit request headers from original request - only those that came from the user, Content-Type is one of them)
  4. Each ESI call ends with 500 response

Response headers:

HTTP/1.1 500 java.nio.file.AccessDeniedException: /file-uploads
Content-Type: text/html
Connection: close
Content-Length: 9372

Response body:

<html>
  <head>
    <title>java.nio.file.AccessDeniedException: /file-uploads</title>
    <style>
        body {
          margin: 0;
          padding: 80px 100px;
          font: 13px "Helvetica Neue", "Lucida Grande", "Arial";
          background: #ECE9E9 -webkit-gradient(linear, 0% 0%, 0% 100%, from(#fff), to(#ECE9E9));
          background: #ECE9E9 -moz-linear-gradient(top, #fff, #ECE9E9);
          background-repeat: no-repeat;
          color: #555;
          -webkit-font-smoothing: antialiased;
        }
        h1, h2, h3 {
          margin: 0;
          font-size: 22px;
          color: #343434;
        }
        h1 em, h2 em {
          padding: 0 5px;
          font-weight: normal;
        }
        h1 {
          font-size: 60px;
        }
        h2 {
            margin-top: 10px;
        }
        h3 {
          margin: 5px 0 10px 0;
          padding-bottom: 5px;
          border-bottom: 1px solid #eee;
          font-size: 18px;
        }
        ul {
          margin: 0;
          padding: 0;
        }
        ul li {
          margin: 5px 0;
          padding: 3px 8px;
          list-style: none;
        }
        ul li:hover {
          cursor: pointer;
          color: #2e2e2e;
        }
        p {
          line-height: 1.5;
        }
        a {
          color: #555;
          text-decoration: none;
        }
        a:hover {
          color: #303030;
        }
        #stacktrace {
            margin-top: 15px;
        }
        .directory h1 {
          margin-bottom: 15px;
          font-size: 18px;
        }
    </style>
  </head>
  <body>
    <div id="wrapper">
      <h1>Matron!</h1>
      <h2><em>500</em> java.nio.file.AccessDeniedException: /file-uploads</h2>
      <ul id="stacktrace"><li>io.vertx.core.file.impl.FileSystemImpl$11.perform(FileSystemImpl.java:678)</li><li>io.vertx.core.file.impl.FileSystemImpl$11.perform(FileSystemImpl.java:660)</li><li>io.vertx.core.file.impl.FileSystemImpl.mkdirsBlocking(FileSystemImpl.java:248)</li><li>io.vertx.ext.web.handler.impl.BodyHandlerImpl$BHandler.makeUploadDir(BodyHandlerImpl.java:171)</li><li>io.vertx.ext.web.handler.impl.BodyHandlerImpl$BHandler.<init>(BodyHandlerImpl.java:138)</li><li>io.vertx.ext.web.handler.impl.BodyHandlerImpl.handle(BodyHandlerImpl.java:72)</li><li>io.vertx.ext.web.handler.impl.BodyHandlerImpl.handle(BodyHandlerImpl.java:42)</li><li>io.vertx.reactivex.ext.web.handler.BodyHandler.handle(BodyHandler.java:74)</li><li>io.vertx.reactivex.ext.web.handler.BodyHandler.handle(BodyHandler.java:37)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:155)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:153)</li><li>io.vertx.ext.web.impl.RouteImpl.handleContext(RouteImpl.java:219)</li><li>io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:120)</li><li>io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:133)</li><li>io.vertx.ext.web.handler.impl.CookieHandlerImpl.handle(CookieHandlerImpl.java:66)</li><li>io.vertx.ext.web.handler.impl.CookieHandlerImpl.handle(CookieHandlerImpl.java:42)</li><li>io.vertx.reactivex.ext.web.handler.CookieHandler.handle(CookieHandler.java:73)</li><li>io.vertx.reactivex.ext.web.handler.CookieHandler.handle(CookieHandler.java:36)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:155)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:153)</li><li>io.vertx.ext.web.impl.RouteImpl.handleContext(RouteImpl.java:219)</li><li>io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:120)</li><li>io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:133)</li><li>io.vertx.reactivex.ext.web.RoutingContext.next(RoutingContext.java:128)</li><li>io.knotx.server.SupportedMethodsAndPathsHandler.handle(SupportedMethodsAndPathsHandler.java:52)</li><li>io.knotx.server.SupportedMethodsAndPathsHandler.handle(SupportedMethodsAndPathsHandler.java:27)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:155)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:153)</li><li>io.vertx.ext.web.impl.RouteImpl.handleContext(RouteImpl.java:219)</li><li>io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:120)</li><li>io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:133)</li><li>io.vertx.reactivex.ext.web.RoutingContext.next(RoutingContext.java:128)</li><li>io.knotx.server.KnotxHeaderHandler.handle(KnotxHeaderHandler.java:41)</li><li>io.knotx.server.KnotxHeaderHandler.handle(KnotxHeaderHandler.java:23)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:155)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:153)</li><li>io.vertx.ext.web.impl.RouteImpl.handleContext(RouteImpl.java:219)</li><li>io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:120)</li><li>io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:133)</li><li>io.vertx.ext.web.handler.impl.LoggerHandlerImpl.handle(LoggerHandlerImpl.java:178)</li><li>io.vertx.ext.web.handler.impl.LoggerHandlerImpl.handle(LoggerHandlerImpl.java:47)</li><li>io.vertx.reactivex.ext.web.handler.LoggerHandler.handle(LoggerHandler.java:73)</li><li>io.vertx.reactivex.ext.web.handler.LoggerHandler.handle(LoggerHandler.java:36)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:155)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:153)</li><li>io.vertx.ext.web.impl.RouteImpl.handleContext(RouteImpl.java:219)</li><li>io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:120)</li><li>io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:133)</li><li>io.vertx.ext.web.impl.RouterImpl.accept(RouterImpl.java:79)</li><li>io.vertx.reactivex.ext.web.Router.accept(Router.java:94)</li><li>io.knotx.server.KnotxServerVerticle.routeSafe(KnotxServerVerticle.java:181)</li><li>io.knotx.server.KnotxServerVerticle.lambda$start$8(KnotxServerVerticle.java:164)</li><li>io.vertx.reactivex.core.http.HttpServer$1.handle(HttpServer.java:111)</li><li>io.vertx.reactivex.core.http.HttpServer$1.handle(HttpServer.java:109)</li><li>io.vertx.core.http.impl.Http1xServerConnection.processMessage(Http1xServerConnection.java:453)</li><li>io.vertx.core.http.impl.Http1xServerConnection.handleMessage(Http1xServerConnection.java:144)</li><li>io.vertx.core.http.impl.HttpServerImpl$ServerHandlerWithWebSockets.handleMessage(HttpServerImpl.java:666)</li><li>io.vertx.core.http.impl.HttpServerImpl$ServerHandlerWithWebSockets.handleMessage(HttpServerImpl.java:619)</li><li>io.vertx.core.net.impl.VertxHandler.lambda$channelRead$1(VertxHandler.java:146)</li><li>io.vertx.core.impl.ContextImpl.lambda$wrapTask$2(ContextImpl.java:337)</li><li>io.vertx.core.impl.ContextImpl.executeFromIO(ContextImpl.java:195)</li><li>io.vertx.core.net.impl.VertxHandler.channelRead(VertxHandler.java:144)</li><li>io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)</li><li>io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)</li><li>io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)</li><li>io.vertx.core.http.impl.HttpServerImpl$Http2UpgradeHandler.channelRead(HttpServerImpl.java:968)</li><li>io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)</li><li>io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)</li><li>io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)</li><li>io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)</li><li>io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)</li><li>io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)</li><li>io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)</li><li>io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)</li><li>io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)</li><li>io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)</li><li>io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)</li><li>io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)</li><li>io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:141)</li><li>io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)</li><li>io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)</li><li>io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)</li><li>io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)</li><li>io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)</li><li>io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)</li><li>java.lang.Thread.run(Thread.java:748)</li></ul>
    </div>
  </body>
</html>

Expected behavior

ESI calls are regular GET requests. The only difference in comparison to other situations is the fact that sub-requests inherit parent request headers, so CDN sends GET with Content-Type: application/x-www-form-urlencoded.

knot.x should either reject such request with 400 status code or ignore Content-Type header sent with GET request, as it doesn't make any sense.

Screenshots

N/A

Additional context

The following curl command was enough to reproduce the problem:

$ curl localhost:8092/path/to/esi/snippet.html -H "Content-Type: application/x-www-form-urlencoded"

Those 500s are visible in knotx-access.log file, but knotx.log stays empty despite of the fact I increased log level to DEBUG.

10.251.35.221 - - [Mon, 14 Jan 2019 22:59:25 GMT] "GET /path/to/esi/snippet.html HTTP/1.1" 500 8931 "http://example.org/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36"

It could be related to #321

Possible workaround - "sanitize" HTTP request before it gets processed by knot.x and remove Content-Type from GET requests.