search

spring-cloud / spring-cloud-gateway

An API Gateway built on Spring Framework and Spring Boot providing routing and more.
http://cloud.spring.io
Apache License 2.0
4.55k stars 3.33k forks source link

[Spring Cloud Gateway MVC] Multipart data missing #3527

Open lukaszpy opened 2 months ago

lukaszpy commented 2 months ago

Describe the bug After routing multipart/form-data through gateway mvc, downstream service receive request but without multipart file.

I looked into code of gateway and in RestClientProxyExchange we have 

@Override
    public ServerResponse exchange(Request request) {
        return restClient.method(request.getMethod()).uri(request.getUri())
                .headers(httpHeaders -> httpHeaders.putAll(request.getHeaders()))
                .body(outputStream -> copyBody(request, outputStream))
                .exchange((clientRequest, clientResponse) -> doExchange(request, clientResponse), false);
    }

there is not copy multipart data from original request.

Example route:

@Bean
    fun dataStorageFileUpload(): RouterFunction<ServerResponse> {
        val servicesUrl = services.url!!
        Objects.requireNonNull(servicesUrl["dataStorageService"], "Data storage service url is empty")
        return GatewayRouterFunctions.route("dataStorageFileUpload")
            .PUT("/storage/file/**", http(servicesUrl["dataStorageService"]))
            .before(BeforeFilterFunctions.stripPrefix(1))
            .filter(HandlerFilterFunction.ofRequestProcessor(authenticationPopular))
            .after(AfterFilterFunctions.removeResponseHeader(GROUP_HEADER))
            .after(AfterFilterFunctions.removeResponseHeader(HttpHeaders.AUTHORIZATION))
            .after(AfterFilterFunctions.removeResponseHeader(TOKEN_HEADER))
            .build()
    }
dgradecak commented 2 months ago

@lukaszpy do you have spring security configured? I had the smae behavior when using spring security, did not dig deeper to understand the cause. For now by removing spring security the body is processed correctly, specially when using spring-cloud-gateway-server-mvc

lukaszpy commented 2 months ago

@dgradecak yes I have spring sec, as validate tokens, and checking roles for path

spencergibb commented 2 months ago

If you'd like us to spend some time investigating, please take the time to provide a complete, minimal, verifiable sample (something that we can unzip attached to this issue or git clone, build, and deploy) that reproduces the problem.

spring-cloud-issues commented 1 month ago

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

sapo-di commented 1 month ago

We had the same issue.

I can provide an example here: https://github.com/sapo-di/gateway-mvc-test/

You need to run the ProxyApplication and can use the forward-mutlipart.http for an example request against http://localhost:8080/multipart

The produces exception is:

2024-10-08T10:54:12.672+02:00 ERROR 266119 --- [proxy] [nio-8080-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://localhost:8081/incoming": insufficient data written] with root cause

java.io.IOException: insufficient data written
    at java.base/sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.close(HttpURLConnection.java:3869) ~[na:na]
    at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1622) ~[na:na]
    at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589) ~[na:na]
    at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529) ~[na:na]
    at org.springframework.http.client.SimpleClientHttpRequest.executeInternal(SimpleClientHttpRequest.java:88) ~[spring-web-6.1.13.jar:6.1.13]
    at org.springframework.http.client.AbstractStreamingClientHttpRequest.executeInternal(AbstractStreamingClientHttpRequest.java:70) ~[spring-web-6.1.13.jar:6.1.13]
    at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:66) ~[spring-web-6.1.13.jar:6.1.13]
    at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:889) ~[spring-web-6.1.13.jar:6.1.13]
    at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:740) ~[spring-web-6.1.13.jar:6.1.13]
    at org.springframework.cloud.gateway.mvc.ProxyExchange.exchange(ProxyExchange.java:347) ~[spring-cloud-gateway-mvc-4.1.4.jar:4.1.4]
    at org.springframework.cloud.gateway.mvc.ProxyExchange.post(ProxyExchange.java:308) ~[spring-cloud-gateway-mvc-4.1.4.jar:4.1.4]
    at org.example.proxy.ForwardMultipartController.forwardMultipart(ForwardMultipartController.java:15) ~[classes/:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:255) ~[spring-web-6.1.13.jar:6.1.13]
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:188) ~[spring-web-6.1.13.jar:6.1.13]
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118) ~[spring-webmvc-6.1.13.jar:6.1.13]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:926) ~[spring-webmvc-6.1.13.jar:6.1.13]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:831) ~[spring-webmvc-6.1.13.jar:6.1.13]
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-6.1.13.jar:6.1.13]
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1089) ~[spring-webmvc-6.1.13.jar:6.1.13]
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:979) ~[spring-webmvc-6.1.13.jar:6.1.13]
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014) ~[spring-webmvc-6.1.13.jar:6.1.13]
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:914) ~[spring-webmvc-6.1.13.jar:6.1.13]
    at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:590) ~[tomcat-embed-core-10.1.30.jar:6.0]
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885) ~[spring-webmvc-6.1.13.jar:6.1.13]
    at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658) ~[tomcat-embed-core-10.1.30.jar:6.0]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:195) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) ~[tomcat-embed-websocket-10.1.30.jar:10.1.30]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-6.1.13.jar:6.1.13]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.13.jar:6.1.13]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-6.1.13.jar:6.1.13]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.13.jar:6.1.13]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.springframework.web.filter.ServerHttpObservationFilter.doFilterInternal(ServerHttpObservationFilter.java:113) ~[spring-web-6.1.13.jar:6.1.13]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.13.jar:6.1.13]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-6.1.13.jar:6.1.13]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.13.jar:6.1.13]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:483) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:384) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:905) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1741) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) ~[tomcat-embed-core-10.1.30.jar:10.1.30]
    at java.base/java.lang.Thread.run(Thread.java:840) ~[na:na]
philwilkinson commented 4 weeks ago

I am also having this issue. With trace level logging on I can see ALL the multipart form data arriving in the gateway, but it is not passed onto the downstream service. I am using spring security, TokenRelay filter. I've tried it with spring-cloud-starter-gateway-mvc 4.1.4 and 4.1.5

I have narrowed it down to the TokenRelay= filter, without it the multi-part form data is passed to the down steam service (with no bearer token), with it there is a bearer token and no multi-part form data.

The problem is with TokenRelay in that it calls OAuth2AuthorizedClient authorizedClient = clientManager.authorize(authorizeRequest); which calls.

HttpServletRequest servletRequest = getHttpServletRequestOrDefault(authorizeRequest.getAttributes());
String scope = servletRequest.getParameter(OAuth2ParameterNames.SCOPE);

in DefaultOAuth2AuthorizedClientManager.DefaultContextAttributesMapper.apply

When getHttpServletRequestOrDefault returns it returns a HttpServletRequest, but not the 'GatewayMultipartHttpServletRequest', which if it did might prevent the servletRequest.getParameter(OAuth2ParameterNames.SCOPE) which ultimately calls org.apache.catalina.connector.Request.parseParameters which parses the multi-part form data from an InputStream and effectively loses the multi-part form data.

My fix was to locally configure the OAuth2AuthorizedClientManager' with a customised DefaultContextAttributesMapper that did not call servletRequest.getParameter(OAuth2ParameterNames.SCOPE); if the servletRequest.getContentType() was a multipart request.