Missing Content-Length Header And HTTP Forwarding Errors

[lewright12]

Firstly, Thanks for the tool, well done.

While working with the mock-server in forwarding mode, I encountered an API call that we had to make that returned an XML document without a Content-Length header. This caused mock server to not respond as shown in the following example. (Hopefully this may help someone else.)

Mock Server Code:

package com.lew;
import org.mockserver.client.MockServerClient;
import org.mockserver.integration.ClientAndServer;
import org.mockserver.model.HttpForward;
import static org.mockserver.model.HttpForward.forward;
import static org.mockserver.model.HttpRequest.request;

public class MockClient {
    private final HttpForward forward;
    private final MockServerClient mockServerClient;
    public MockClient() {
        this.mockServerClient = new ClientAndServer(10000);
        this.forward = forward()
                .withHost("localhost")
                .withPort(11000)
                .withScheme(HttpForward.Scheme.HTTP);
        mockServerClient.when(request().withMethod("GET"))
                .forward(forward);
    }
    public static void main(String[] args) throws Exception {
        new MockClient();
    }
}

Simple Python Server To demonstrate:

#!/usr/bin/python3
from http.server import BaseHTTPRequestHandler, HTTPServer

class HackedBaseHTTPRequestHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        f = open("./some.xml", 'r')
        message = f.read()
        f.close()
        print(str(self.requestline) + " - Length: " + str(len(message)))
        self.send_response(200)
        self.send_header("Content-Type", "application/xml")
        if "withheader" in self.requestline:
            self.send_header("Content-Length", str(len(message)))
        self.end_headers()
        self.wfile.write(bytes(message, "utf8"))
        return

if __name__ == '__main__':
    httpd = HTTPServer(('127.0.0.1', 11000), HackedBaseHTTPRequestHandler)
    httpd.serve_forever()

Curl requests showing the error:

luther@lwright-7490:~/$ curl -v http://localhost:10000/withoutheader
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 10000 (#0)
> GET /withoutheader HTTP/1.1
> Host: localhost:10000
> User-Agent: curl/7.58.0
> Accept: */*
> 
^C
luther@lwright-7490:~/$ curl -v http://localhost:10000/withheader
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 10000 (#0)
> GET /withheader HTTP/1.1
> Host: localhost:10000
> User-Agent: curl/7.58.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: BaseHTTP/0.6 Python/3.6.7
< Date: Tue, 15 Jan 2019 16:17:51 GMT
< Content-Type: application/xml
< Content-Length: 161
< connection: keep-alive
< 
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<MockedDocument>
  <SomeResponse>
    <ElementOne>Hello</ElementOne>
  </SomeResponse>
</MockedDocument>
* Connection #0 to host localhost left intact
luther@lwright-7490:~/$

Reviewing the HTTP RFC 2616 the server SHOULD send the Content-Length header. But digging in a little deeper it seems that the server can just close the connection and not send the Content-Length. I was looking at section 4.4 ( server closing the connection ) and section 14.13 ( Content-Length definition ). Perhaps my interpretation is wrong, please let me know if so.

My stack trace from Mock Server:

11:17:15.827 [pool-3-thread-10] ERROR org.mockserver.mock.HttpStateHandler - org.mockserver.client.netty.SocketConnectionException: Channel set as inactive before valid response has been received
java.util.concurrent.ExecutionException: org.mockserver.client.netty.SocketConnectionException: Channel set as inactive before valid response has been received
    at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:500) ~[guava-20.0.jar:na]
    at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:459) ~[guava-20.0.jar:na]
    at com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:76) ~[guava-20.0.jar:na]
    at org.mockserver.mock.action.ActionHandler$13.run(ActionHandler.java:244) ~[mockserver-core-5.5.1.jar:5.5.1]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_181]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_181]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_181]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_181]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_181]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_181]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]
Caused by: org.mockserver.client.netty.SocketConnectionException: Channel set as inactive before valid response has been received
    at org.mockserver.client.netty.HttpClientConnectionHandler.updatePromise(HttpClientConnectionHandler.java:18) ~[mockserver-core-5.5.1.jar:5.5.1]
    at org.mockserver.client.netty.HttpClientConnectionHandler.channelInactive(HttpClientConnectionHandler.java:24) ~[mockserver-core-5.5.1.jar:5.5.1]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1429) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:947) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:826) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) ~[netty-common-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) ~[netty-common-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:474) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) ~[netty-common-4.1.32.Final.jar:4.1.32.Final]
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.32.Final.jar:4.1.32.Final]
    ... 1 common frames omitted

[jamesdbloom]

Thank you for the detailed and excellent bug report. This is now fixed and I've added a test to cover this exact scenario. It is now available in the SNAPSHOT version: http://mock-server.com/where/maven_central.html#sonatype_snapshot and will be in a release in the next week or so.