mkopylec / charon-spring-boot-starter

Reverse proxy implementation in form of a Spring Boot starter.
Apache License 2.0
245 stars 55 forks source link

Multipart file uploads supported? #34

Closed joerwin closed 6 years ago

joerwin commented 7 years ago

I am trying to utilize your proxy implementation to add a security layer between my UI and multiple backend REST services. I have been able to handle all OPTIONS requests as well as all GET requests. I am wondering how to manage proxying a Multipart file upload POST request.

I have found that your implementation utilises the Netty4ClientHttpRequestFactory wich in its documentation indicates "this implementation consistently closes the HTTP connection on each request". So I have overridden the HttpClientProvider to utilize the HttpComponentsAsyncClientHttpRequestFactory instead to allow for keep alive connections.

We have an HAProxy instance between my implementation and the actual REST services. Becuase the security layer has its own DNS host name entry I have also overridden the ReverseProxyFilter.addForwardHeaders to set the HOST header correctly (from the mappingProperties) so that the host name HAPRoxy ACLs will function correctly.

The service I am proxying my requests to continues to return a 400 BAD Request response to my requests die to the body not containing the boundry.

is there special configuration required to handle mutlipart file uploads?

mkopylec commented 6 years ago

Hi, the newest version of Charon uses Apache HTTP client for forwarding requests: https://github.com/mkopylec/charon-spring-boot-starter/blob/master/src/main/java/com/github/mkopylec/charon/core/http/HttpClientProvider.java I haven't tested multipart file uploads, maybe try something like: https://stackoverflow.com/questions/7003045/attempting-to-test-rest-service-with-multipart-file Maybe you are sending an incorrect HTTP request or the receiving service behind the HAProxy is incorrectly configured. Let me know if you have found any solution for that.

joerwin commented 6 years ago

I ended up identifying a couple of issues. I was using spring and it was inserting the HiddenHttpMethodFilter into the filter chain and this was invalidating the input stream and causing the body forwarded to be empty which was causing bad requests. The next issue that I had to solve was that the restTemplate exchange would read in the entire file part into memory causing OOM issues in my proxy. To remedy this I check the request to see if it is multipart (ServletFileUpload.isMultipartContent(httpServletRequest)) and then I manage the multipart myself using an HttpUrlConnection. This allowed me to write directly to the output stream of the open connection keeping memory consumption reasonable.

        httpConn.setRequestProperty("Transfer-Encoding","chunked");
        httpConn.setChunkedStreamingMode(1);
        httpConn.connect();

        IOUtils.copy(request.getInputStream(), httpConn.getOutputStream());