perwendel / spark

A simple expressive web framework for java. Spark has a kotlin DSL https://github.com/perwendel/spark-kotlin
Apache License 2.0
9.64k stars 1.56k forks source link

Large multipart upload response problem #1059

Open hakucuk opened 5 years ago

hakucuk commented 5 years ago

Given the codes below and trying to upload large file than 10KB, we can not get the response if we call /upload endpoint sometimes. In version 2.7.2 the behaviour is random. sometimes we got exception, sometimes no response. In version 2.8 it is always no response. Is there a way to get an exception message?

 public static void main(String[] args) {
      post("/upload", (request, response) -> {
      request.attribute("org.eclipse.jetty.multipartConfig",
                new MultipartConfigElement(System.getProperty("java.io.tmpdir"), 10000, 10000, 10000));
        InputStream i  =request.raw().getPart("test").getInputStream();
        i.close();
        return "tested";
    });
}
valtterip commented 5 years ago

Have you tried tweaking the MultipartConfigElement params? I would assume setting the max request size larger than the max filesize would be a good idea. Also, the treshold seems quite high here, set it to 1024 or something. Read more at https://docs.oracle.com/javaee/6/api/javax/servlet/MultipartConfigElement.html

But not getting an exception or anything sounds like things are happening already before reaching the route. Do you have a proxy limiting the request size? Do you have a request timeout set in client? The network tab in chrome devtools is your friend.

hakucuk commented 5 years ago

I get same inconsistent behavior with the config below


public static void main(String[] args) {
        post("/upload", (request, response) -> {
            request.attribute("org.eclipse.jetty.multipartConfig",
                    new MultipartConfigElement(System.getProperty("java.io.tmpdir"), 10000, 20000, 1024));
            InputStream i  =request.raw().getPart("test").getInputStream();
            i.close();
            return "tested";
        });
    }

I don't have any proxy. I don't set any timeout explicitly. I am testing with postman. These are the two screenshots for the same request.

image

image

toyg commented 5 years ago

a 500 should surface something in logs, have you enabled DEBUG on the root logger?

Lloyd-Pottiger commented 3 years ago

I test the code memtioned in issue desrciption, with command

curl -F 'test=@CCPP.csv' http://127.0.0.1:4567/upload

where CCPP.csv is file that larger than 10KB. Having tried a lot of times, and each time the log is the same:

ERROR spark.http.matching.GeneralError - 
java.lang.IllegalStateException: Multipart Mime part file exceeds max filesize
    at org.eclipse.jetty.util.MultiPartInputStreamParser$MultiPart.write(MultiPartInputStreamParser.java:168)
    at org.eclipse.jetty.util.MultiPartInputStreamParser.parse(MultiPartInputStreamParser.java:790)
    at org.eclipse.jetty.util.MultiPartInputStreamParser.getParts(MultiPartInputStreamParser.java:475)
    at org.eclipse.jetty.server.MultiParts$MultiPartsUtilParser.getParts(MultiParts.java:112)
    at org.eclipse.jetty.server.Request.getParts(Request.java:2404)
    at org.eclipse.jetty.server.Request.getParts(Request.java:2388)
    at org.eclipse.jetty.server.Request.getPart(Request.java:2378)
    at javax.servlet.http.HttpServletRequestWrapper.getPart(HttpServletRequestWrapper.java:386)
    at spark.MyTest.lambda$main$0(MyTest.java:12)
    at spark.RouteImpl$1.handle(RouteImpl.java:72)
    at spark.http.matching.Routes.execute(Routes.java:61)
    at spark.http.matching.MatcherFilter.doFilter(MatcherFilter.java:136)
    at spark.embeddedserver.jetty.JettyHandler.doHandle(JettyHandler.java:50)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1586)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
    at org.eclipse.jetty.server.Server.handle(Server.java:516)
    at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
    at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
    at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
    at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
    at java.lang.Thread.run(Thread.java:748)

And the result got in shell is <html><body><h2>500 Internal Server Error</h2></body></html>, which is the result expected.

Also, I tests with command

curl -F 'test=@test.csv' http://127.0.0.1:4567/upload

where test.csv is file that samller than 10KB. The result got in shell is tested and no error in log as expected.

Therefore, I can get an expected exception message in version 2.9.4, and I think the issue can be closed.