gateway-mvc: chunked responses with status 500 are damaged

[jgebauer]

Describe the bug

When using default tomcat servlet container, 500 errors that are produced in the downstream service are not correctly transfered to the client.

Actual behaviour

curl localhost:8080/gateway/api
curl: (56) Illegal or missing hexadecimal sequence in chunked-encoding

Expected behaviour

curl localhost:8080/gateway/api
{"timestamp":"2021-05-09T14:37:51.758+00:00","status":500,"error":"Internal Server Error","message":"","path":"/api"}

This issue has been already reported in #1660. To sum up the findings:

• the error can be only reproduce when using tomcat servlet container

• the error can be reproduced with a plain mvc controller when setting 'chunked' and 'close' response headers

  @GetMapping("/proxy500")
  public ResponseEntity<?> proxy500(ProxyExchange<byte[]> proxy) {
     MultiValueMap<String, String> multiValueMap = new HttpHeaders();
     multiValueMap.put("Transfer-Encoding", Collections.singletonList("chunked"));
      multiValueMap.put("Connection", Collections.singletonList("close"));
      return new ResponseEntity<>("{", multiValueMap, HttpStatus.INTERNAL_SERVER_ERROR);
  
  }

• so https://github.com/spring-projects/spring-framework/issues/24699 has been opened to report this issue
  • the issue has been forwarded to the tomcat bug tracker https://bz.apache.org/bugzilla/show_bug.cgi?id=65201
  • the tomcat issue tracker rejected the error, since setting "close" and "chunked" headers by the application without handling them properly is considered an application bug

That's why I am posting here again. Maybe there are some response headers that should not be forwarded upstream by the gateway implementation since those headers are meant for the connection between the gateway and the downstream server.

Not setting those headers will cause the spring mvc framework on the gateway server to determine correct response headers that fit to the connection between client and gateway right?

Sample The sample gateway-forward-error-bug.zip showcases the error by providing a gateway api at http://localhost:8080/gateway/api which is proxied to http://localhost:8080/api that always produces a 500 error. It is expected that the message details are being transfered back to the client calling the gateway endpoint.

[imbyungjun]

I resolved this issue with adding a servlet filter inspired by RemoveHopByHopHeadersFilter. As written in spring cloud docs, hop by hop headers have to be removed on proxy server as defined in ietf.

@Component
public class RemoveHopByHopHeadersServletFilter implements Filter {
    private static final Set<String> HEADERS_REMOVED_ON_REQUEST = Set.of("connection", "keep-alive", "transfer-encoding", "te", "trailer", "proxy-authorization", "proxy-authenticate", "x-application-context", "upgrade");

    @Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
        chain.doFilter(request, new HttpServletResponseWrapper(response) {
            @Override
            public void addHeader(String name, String value) {
                if (!HEADERS_REMOVED_ON_REQUEST.contains(name.toLowerCase())) {
                    super.addHeader(name, value)
                }
            }
        });
    }
}

After adding the filter, response is handled properly whether the status code is 200 or 400.

❯ curl -v --raw -X POST -d '{}' http://localhost:8080/proxy/api/foo
Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
> POST /proxy/api/foo HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Length: 2
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 2 out of 2 bytes
< HTTP/1.1 400
< Date: Tue, 07 Sep 2021 12:54:59 GMT
< Content-Type: application/json
< Transfer-Encoding: chunked
< Connection: close
<
f
{"key":"value"}
0

* Closing connection 0