search

mitre / HTTP-Proxy-Servlet

Smiley's HTTP Proxy implemented as a Java servlet
Apache License 2.0
1.46k stars 553 forks source link

POST Multipart request doesn't work in Proxy Servlet #186

Closed SelimEmre closed 2 years ago

SelimEmre commented 3 years ago

I'm using this library in my project. I'm using GET, PUT, DELETE, POST requests with Proxy Servlet. All of them working properly. But POST request with form-data doesn't working properly. If I try without SSL, it's working properly. Here are my request details:

var data = new FormData();
data.append("file", fileInput.files[0], "ffmpeg_test.mp4");
 data.append("file_info", "test");

var xhr = new XMLHttpRequest();
xhr.withCredentials = true;

xhr.addEventListener("readystatechange", function() {
  if(this.readyState === 4) {
    console.log(this.responseText);
  }
});

xhr.open("POST", "https://domain:5443/rest/request?_path=WebRTCAppEE/rest/v2/vods/create?name=ForBiggerFun.mp4");

xhr.send(data);

Here are my logs:


2020-12-20 19:33:07,659 [https-openssl-apr-0.0.0.0-5443-exec-16] INFO  o.a.http.impl.execchain.RetryExec - I/O exception (org.apache.catalina.connector.ClientAbortException) caught when processing request to {}->http://localhost:5080: java.io.IOException: Stream reset
2020-12-20 19:33:07,681 [http-apr-0.0.0.0-5080-exec-6] ERROR o.a.c.c.C.[.[.0.0.0.[.[jersey-serlvet] - Servlet.service() for servlet [jersey-serlvet] in context with path [/WebRTCAppEE] threw exception [org.glassfish.jersey.server.ContainerException: org.apache.catalina.connector.ClientAbortException: java.io.EOFException: Unexpected EOF read on the socket] with root cause
java.io.EOFException: Unexpected EOF read on the socket
    at org.apache.coyote.http11.Http11InputBuffer.fill(Http11InputBuffer.java:807)
    at org.apache.coyote.http11.Http11InputBuffer.access$300(Http11InputBuffer.java:42)
    at org.apache.coyote.http11.Http11InputBuffer$SocketInputBuffer.doRead(Http11InputBuffer.java:1175)
    at org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:140)
    at org.apache.coyote.http11.Http11InputBuffer.doRead(Http11InputBuffer.java:263)
    at org.apache.coyote.Request.doRead(Request.java:585)
    at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:344)
    at org.apache.catalina.connector.InputBuffer.checkByteBufferEof(InputBuffer.java:663)
    at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:370)
    at org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:183)
    at org.glassfish.jersey.message.internal.EntityInputStream.read(EntityInputStream.java:79)
    at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream.read(ReaderInterceptorExecutor.java:273)
    at org.jvnet.mimepull.MIMEParser.fillBuf(MIMEParser.java:415)
    at org.jvnet.mimepull.MIMEParser.readBody(MIMEParser.java:191)
    at org.jvnet.mimepull.MIMEParser.access$600(MIMEParser.java:38)
    at org.jvnet.mimepull.MIMEParser$MIMEEventIterator.next(MIMEParser.java:140)
    at org.jvnet.mimepull.MIMEParser$MIMEEventIterator.next(MIMEParser.java:102)
    at org.jvnet.mimepull.MIMEMessage.makeProgress(MIMEMessage.java:200)
    at org.jvnet.mimepull.MIMEMessage.parseAll(MIMEMessage.java:160)
    at org.jvnet.mimepull.MIMEMessage.getAttachments(MIMEMessage.java:86)
    at org.glassfish.jersey.media.multipart.internal.MultiPartReaderClientSide.getMimeParts(MultiPartReaderClientSide.java:248)
    at org.glassfish.jersey.media.multipart.internal.MultiPartReaderClientSide.readMultiPart(MultiPartReaderClientSide.java:207)
    at org.glassfish.jersey.media.multipart.internal.MultiPartReaderServerSide.readMultiPart(MultiPartReaderServerSide.java:67)
    at org.glassfish.jersey.media.multipart.internal.MultiPartReaderClientSide.readFrom(MultiPartReaderClientSide.java:159)
    at org.glassfish.jersey.media.multipart.internal.MultiPartReaderClientSide.readFrom(MultiPartReaderClientSide.java:69)
    at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$TerminalReaderInterceptor.invokeReadFrom(ReaderInterceptorExecutor.java:233)
    at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$TerminalReaderInterceptor.aroundReadFrom(ReaderInterceptorExecutor.java:212)
    at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor.proceed(ReaderInterceptorExecutor.java:132)
    at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundReadFrom(MappableExceptionWrapperInterceptor.java:49)
    at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor.proceed(ReaderInterceptorExecutor.java:132)
    at org.glassfish.jersey.message.internal.MessageBodyFactory.readFrom(MessageBodyFactory.java:1072)
    at org.glassfish.jersey.message.internal.InboundMessageContext.readEntity(InboundMessageContext.java:885)
    at org.glassfish.jersey.message.internal.InboundMessageContext.readEntity(InboundMessageContext.java:819)
    at org.glassfish.jersey.server.ContainerRequest.readEntity(ContainerRequest.java:237)
    at org.glassfish.jersey.media.multipart.internal.FormDataParamValueParamProvider$ValueProvider.getEntity(FormDataParamValueParamProvider.java:84)
    at org.glassfish.jersey.media.multipart.internal.FormDataParamValueParamProvider$FormDataParamValueProvider.apply(FormDataParamValueParamProvider.java:235)
    at org.glassfish.jersey.media.multipart.internal.FormDataParamValueParamProvider$FormDataParamValueProvider.apply(FormDataParamValueParamProvider.java:222)
    at org.glassfish.jersey.server.spi.internal.ParamValueFactoryWithSource.apply(ParamValueFactoryWithSource.java:50)
    at org.glassfish.jersey.server.spi.internal.ParameterValueHelper.getParameterValues(ParameterValueHelper.java:64)
    at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$AbstractMethodParamInvoker.getParamValues(JavaResourceMethodDispatcherProvider.java:109)
    at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80)
    at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232)
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)
    at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.catalina.filters.ExpiresFilter.doFilter(ExpiresFilter.java:1230)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at io.antmedia.filter.IPFilter.doFilter(IPFilter.java:24)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at io.antmedia.filter.JWTFilter.doFilter(JWTFilter.java:37)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.red5.logging.LoggerContextFilter.doFilter(LoggerContextFilter.java:84)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at io.antmedia.filter.CorsHeaderFilter.handleSimpleCORS(CorsHeaderFilter.java:63)
    at org.apache.catalina.filters.CorsFilter.doFilter(CorsFilter.java:158)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:544)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:616)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:831)
    at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2075)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.base/java.lang.Thread.run(Thread.java:834)
2020-12-20 19:33:07,684 [https-openssl-apr-0.0.0.0-5443-exec-16] ERROR o.a.c.c.C.[.[.0.0.0.[.[RequestDispatherFilter] - Servlet.service() for servlet [RequestDispatherFilter] in context with path [] threw exception
org.apache.http.client.ClientProtocolException: null
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
    at org.mitre.dsmiley.httpproxy.ProxyServlet.doExecute(ProxyServlet.java:386)
    at org.mitre.dsmiley.httpproxy.ProxyServlet.service(ProxyServlet.java:328)
    at org.mitre.dsmiley.httpproxy.URITemplateProxyServlet.service(URITemplateProxyServlet.java:140)
    at io.antmedia.console.servlet.ProxyServlet.service(ProxyServlet.java:22)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:733)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at io.antmedia.console.rest.AuthenticationFilter.doFilter(AuthenticationFilter.java:33)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at io.antmedia.filter.IPFilterDashboard.doFilter(IPFilterDashboard.java:23)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.catalina.filters.CorsFilter.handleNonCORS(CorsFilter.java:364)
    at org.apache.catalina.filters.CorsFilter.doFilter(CorsFilter.java:170)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.red5.logging.LoggerContextFilter.doFilter(LoggerContextFilter.java:84)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:544)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
    at org.apache.coyote.http2.StreamProcessor.service(StreamProcessor.java:368)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
    at org.apache.coyote.http2.StreamProcessor.process(StreamProcessor.java:71)
    at org.apache.coyote.http2.StreamRunnable.run(StreamRunnable.java:35)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.http.client.NonRepeatableRequestException: Cannot retry request with a non-repeatable request entity
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:108)
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
    ... 41 common frames omitted
Caused by: org.apache.catalina.connector.ClientAbortException: java.io.IOException: Stream reset
    at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:348)
    at org.apache.catalina.connector.InputBuffer.checkByteBufferEof(InputBuffer.java:663)
    at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:370)
    at org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:183)
    at org.apache.http.entity.InputStreamEntity.writeTo(InputStreamEntity.java:140)
    at org.apache.http.impl.execchain.RequestEntityProxy.writeTo(RequestEntityProxy.java:121)
    at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156)
    at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:152)
    at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238)
    at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
    ... 43 common frames omitted
Caused by: java.io.IOException: Stream reset
    at org.apache.coyote.http2.Stream$StreamInputBuffer.doRead(Stream.java:1117)
    at org.apache.coyote.Request.doRead(Request.java:585)
    at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:344)
    ... 55 common frames omitted

Here is my environment: Loaded Apache Tomcat Native library [1.2.25] using APR version [1.6.3]. APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true] APR/OpenSSL configuration: useAprConnector [true], useOpenSSL [true]

Tomcat 8.5.58

I have tried the following:

  1. I have already added <UpgradeProtocol overheadWindowUpdateThreshold="-1" overheadDataThreshold="-1" writeTimeout="-1" streamWriteTimeout="-1" streamReadTimeout="-1" maxHeaderSize="8192" maxConcurrentStreams="300" readTimeout="-1" className="org.apache.coyote.http2.Http2Protocol" compressibleMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,application/json" compression="on" compressionMinSize="1024"/> as by referred from here -> https://stackoverflow.com/questions/58118776/apache-commons-fileupload-problems-uploading-files-greater-than-100kb-using-htt

  2. I upgraded Tomcat to latest version(8.5.61). Still doesn't work.

dsmiley commented 3 years ago

I am more of a steward than active maintainer these days, so you'll have to debug this to the root cause; sorry.

tuanvayn commented 3 years ago

I am more of a steward than active maintainer these days, so you'll have to debug this to the root cause; sorry.

I'm facing with this issue. I want to edit something in multipart file then forward to targeturi

tuanvayn commented 3 years ago

i really need help @dsmiley

dsmiley commented 3 years ago

All I can really offer is that someone who can see this issue will likely need to roll up their sleeves with a debugger. It may also be useful to rule out things... like is Tomcat pertinent? Try with Jetty. Is SSL pertinent as the O.P. showed?

SelimEmre commented 3 years ago

Hi @tuanvayn,

I have fixed this issue. My issue was related to HTTP/2 and Content-type. Please try with HTTP/1 and right Content-type parameters.

Lvyc1995 commented 2 years ago

@SelimEmre

Hi @tuanvayn,

I have fixed this issue. My issue was related to HTTP/2 and Content-type. Please try with HTTP/1 and right Content-type parameters.

I encountered a type of problem, prompt: multipart: EOF, can you explain your solution in detail?

SelimEmre commented 2 years ago

Hi @Lvyc1995, I was trying to upload with application/json content type. Please make sure that you are using the right type.