mock-server / mockserver

MockServer enables easy mocking of any system you integrate with via HTTP or HTTPS with clients written in Java, JavaScript and Ruby. MockServer also includes a proxy that introspects all proxied traffic including encrypted SSL traffic and supports Port Forwarding, Web Proxying (i.e. HTTP proxy), HTTPS Tunneling Proxying (using HTTP CONNECT) and SOCKS Proxying (i.e. dynamic port forwarding).
http://mock-server.com
Apache License 2.0
4.57k stars 1.07k forks source link

Response expectations not writing #200

Closed benjaminhkaiser closed 6 years ago

benjaminhkaiser commented 8 years ago

This is the expectation I set up using the Java API:

fMockServer = startClientAndServer(PORT);

fMockServer.when(request()
                    .withMethod("PUT")
                    .withPath(CONTAINER + ADDRESS1.convertToString()))
                    .respond(response()
                                    .withStatusCode(200)
                                    .withHeader(new Header("Location", CONTAINER + ADDRESS2.convertToString()))
                                    .withCookie(COOKIE1)
                                    .withBody(new StringBody("Response 1"))
                                    .withConnectionOptions(
                                                    new ConnectionOptions()
                                                                    .withCloseSocket(Boolean.FALSE)));

With the server running, when I issue a matching request via CURL, I get the debug messages that the expectation was matched, followed by this output:

11:04:54.773 [nioEventLoopGroup-5-4] INFO  o.m.mockserver.MockServerHandler - returning response:

{
  "statusCode" : 200,
  "headers" : [ {
    "name" : "Location",
    "values" : [ "/test-container/11dda2b7db2f269bda60825a22bbd138a02fd297c757814b39b20ef836e2e1fd" ]
  } ],
  "cookies" : [ {
    "name" : "Cookie 1",
    "value" : "AAAAA"
  } ],
  "body" : "Response 1",
  "connectionOptions" : {
    "closeSocket" : false
  }
}

 for request:

{
  "method" : "PUT",
  "path" : "/test-container/ec7be4eda536206a1da23d640754cede0f5cf894f7209d7a6b9400b80326e790",
  "headers" : [ {
    "name" : "User-Agent",
    "values" : [ "curl/7.41.0" ]
  }, {
    "name" : "Host",
    "values" : [ "localhost:1090" ]
  }, {
    "name" : "Accept",
    "values" : [ "*/*" ]
  }, {
    "name" : "Content-Length",
    "values" : [ "0" ]
  } ],
  "keepAlive" : true,
  "secure" : false
}

11:04:54.773 [nioEventLoopGroup-5-4] DEBUG i.n.handler.logging.LoggingHandler - [id: 0x41f3d411, /0:0:0:0:0:0:0:1:9524 => /0:0:0:0:0:0:0:1:1090] FLUSH

When I send a non-matching request, I get:

11:06:51.368 [nioEventLoopGroup-5-5] INFO  o.m.mockserver.MockServerHandler - returning response:

{
  "statusCode" : 404
}

 for request:

{
  "method" : "PUT",
  "path" : "/test-container/blah",
  "headers" : [ {
    "name" : "User-Agent",
    "values" : [ "curl/7.41.0" ]
  }, {
    "name" : "Host",
    "values" : [ "localhost:1090" ]
  }, {
    "name" : "Accept",
    "values" : [ "*/*" ]
  }, {
    "name" : "Content-Length",
    "values" : [ "0" ]
  } ],
  "keepAlive" : true,
  "secure" : false
}

11:06:51.368 [nioEventLoopGroup-5-5] DEBUG i.n.handler.logging.LoggingHandler - [id: 0x8f27b848, /0:0:0:0:0:0:0:1:9525 => /0:0:0:0:0:0:0:1:1090] WRITE, DefaultFullHttpResponse(decodeResult: success, version: HTTP/1.1, content: EmptyByteBufBE)
HTTP/1.1 404 Not Found
Content-Length: 0
Connection: keep-alive, 0B
11:06:51.368 [nioEventLoopGroup-5-5] DEBUG i.n.handler.logging.LoggingHandler - [id: 0x8f27b848, /0:0:0:0:0:0:0:1:9525 => /0:0:0:0:0:0:0:1:1090] FLUSH
11:06:51.368 [nioEventLoopGroup-5-5] DEBUG i.n.handler.logging.LoggingHandler - [id: 0x8f27b848, /0:0:0:0:0:0:0:1:9525 => /0:0:0:0:0:0:0:1:1090] FLUSH
11:06:51.368 [nioEventLoopGroup-5-5] DEBUG i.n.handler.logging.LoggingHandler - [id: 0x8f27b848, /0:0:0:0:0:0:0:1:9525 :> /0:0:0:0:0:0:0:1:1090] INACTIVE
11:06:51.368 [nioEventLoopGroup-5-5] DEBUG i.n.handler.logging.LoggingHandler - [id: 0x8f27b848, /0:0:0:0:0:0:0:1:9525 :> /0:0:0:0:0:0:0:1:1090] UNREGISTERED

The problem is, with the valid request, no response is actually returned! The DEBUG messages show a FLUSH, but no WRITE. With the invalid request, I actually get the 404 response, and you can see the DEBUG messages showing a WRITE before the FLUSH. Why is this happening? I've double checked my syntax against the API docs and the example code multiple times.

jamesdbloom commented 8 years ago

Perhaps this is an issue with the way new ConnectionOptions().withCloseSocket(Boolean.FALSE) is handled.

jamesdbloom commented 8 years ago

The code looks like it is doing a flush, as follows:

        if (closeChannel) {
            ctx.writeAndFlush(response).addListener(ChannelFutureListener.CLOSE);
        } else {
            ctx.write(response);
        }

I'll need to debug this and look into it further.

jamesdbloom commented 6 years ago

Are you still getting the same issue, because I can't reproduce the problem, as follows:

Setup Expectation

import org.mockserver.client.server.MockServerClient;
import org.mockserver.model.ConnectionOptions;

import java.util.concurrent.TimeUnit;

import static org.mockserver.integration.ClientAndServer.startClientAndServer;
import static org.mockserver.model.HttpRequest.request;
import static org.mockserver.model.HttpResponse.response;

public class MockServerTroubleshootingTest {

    private static final int PORT = 1080;

    public static void main(String[] args) throws InterruptedException {
        MockServerClient mockServer = startClientAndServer(PORT);

        mockServer
                .when(
                        request()
                                .withMethod("PUT")
                                .withPath("/test-container/ec7be4eda536206a1da23d640754cede0f5cf894f7209d7a6b9400b80326e790")
                )
                .respond(
                        response()
                                .withStatusCode(200)
                                .withHeader("Location", "/test-container/11dda2b7db2f269bda60825a22bbd138a02fd297c757814b39b20ef836e2e1fd")
                                .withCookie("Cookie 1", "AAAAA")
                                .withBody("Response 1")
                                .withConnectionOptions(
                                        new ConnectionOptions()
                                                .withCloseSocket(Boolean.FALSE))
                );

        TimeUnit.HOURS.sleep(1);
    }
}

Send Request

$ curl -v -X PUT "http://localhost:1080/test-container/ec7be4eda536206a1da23d640754cede0f5cf894f7209d7a6b9400b80326e790"
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 1080 (#0)
> PUT /test-container/ec7be4eda536206a1da23d640754cede0f5cf894f7209d7a6b9400b80326e790 HTTP/1.1
> Host: localhost:1080
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Location: /test-container/11dda2b7db2f269bda60825a22bbd138a02fd297c757814b39b20ef836e2e1fd
< connection: keep-alive
< content-length: 10
< set-cookie: Cookie 1=AAAAA
< 
* Connection #0 to host localhost left intact
Response 1

As demonstrated the response (including its body) is correctly being returned.

Closing ticket, please re-open if you are still have the same issue.

xskerik commented 6 years ago

Hi, it seems I have the same issue. The problem occurs when test is run with the others (more units in one test class). When the test is run isolated, mockserver works fine.

The test fails when RestTemplate is used. When nonblocking client WebClient is used, everything works perfect.

I use:

        <dependency>
            <groupId>org.mock-server</groupId>
            <artifactId>mockserver-netty</artifactId>
            <version>5.3.0</version>
        </dependency>

Expectation:

        new MockServerClient("localhost", 8700)
                .when(
                        request()
                                .withMethod("POST")
                                .withPath("/audiofile/registration")
                                .withQueryStringParameters(
                                        new Parameter("path", "some/path")
                                )
                                .withHeader(new Header(HttpHeaders.AUTHORIZATION, "xxx="))
                )
                .respond(
                        response()
                                .withStatusCode(200)
                                .withHeader(new Header("Content-Type", "application/json"))
                );

Isolated test produce:

2018-06-07 13:49:13,759 INFO nioEventLoopGroup-3-4 HttpStateHandler.infoLog - request:

    {
      "method" : "POST",
      "path" : "/audiofile/registration",
      "queryStringParameters" : {
        "path" : [ "path" ]
      },
      "body" : {
        "type" : "STRING",
        "string" : "{}",
        "contentType" : "text/plain; charset=utf-8"
      },
      "headers" : {
        "Authorization" : [ "Basic xxx=" ],
        "Accept" : [ "application/json" ],
        "X-WebhookTarget" : [ "http://localhost:8091/webhook/results" ],
        "Connection" : [ "Keep-Alive" ],
        "User-Agent" : [ "Apache-HttpClient/4.5.5 (Java/1.8.0_171)" ],
        "Host" : [ "localhost:8700" ],
        "Accept-Encoding" : [ "gzip,deflate" ],
        "Content-Length" : [ "2" ],
        "Content-Type" : [ "application/json" ]
      },
      "keepAlive" : true,
      "secure" : false
    }

 matched expectation:

    {
      "method" : "POST",
      "path" : "/audiofile/registration",
      "queryStringParameters" : {
        "path" : [ "path" ]
      },
      "headers" : {
        "Authorization" : [ "Basic xxx=" ]
      }
    }

2018-06-07 13:49:13,770 INFO nioEventLoopGroup-3-4 HttpStateHandler.infoLog - returning response:

    {
      "statusCode" : 200,
      "headers" : {
        "Content-Type" : [ "application/json" ],
        "connection" : [ "keep-alive" ]
      }
    }

 for request:

    {
      "method" : "POST",
      "path" : "/audiofile/registration",
      "queryStringParameters" : {
        "path" : [ "path" ]
      },
      "body" : {
        "type" : "STRING",
        "string" : "{}",
        "contentType" : "text/plain; charset=utf-8"
      },
      "headers" : {
        "Authorization" : [ "Basic xxx=" ],
        "Accept" : [ "application/json" ],
        "X-WebhookTarget" : [ "http://localhost:8091/webhook/results" ],
        "Connection" : [ "Keep-Alive" ],
        "User-Agent" : [ "Apache-HttpClient/4.5.5 (Java/1.8.0_171)" ],
        "Host" : [ "localhost:8700" ],
        "Accept-Encoding" : [ "gzip,deflate" ],
        "Content-Length" : [ "2" ],
        "Content-Type" : [ "application/json" ]
      },
      "keepAlive" : true,
      "secure" : false
    }

 for response action:

    {
      "statusCode" : 200,
      "headers" : {
        "Content-Type" : [ "application/json" ]
      }
    }

Not isolated test produces only:

2018-06-07 14:00:08,145 INFO nioEventLoopGroup-20-2 HttpStateHandler.infoLog - creating expectation:

    {
      "httpRequest" : {
        "method" : "POST",
        "path" : "/audiofile/registration",
        "queryStringParameters" : {
          "path" : [ "path" ]
        },
        "headers" : {
          "Authorization" : [ "Basic xxx=" ]
        }
      },
      "times" : {
        "remainingTimes" : 0,
        "unlimited" : true
      },
      "timeToLive" : {
        "unlimited" : true
      },
      "httpResponse" : {
        "statusCode" : 200,
        "headers" : {
          "Content-Type" : [ "application/json" ]
        }
      }
    }

And excpetion by client is thrown: 2018-06-07 14:00:08,233 DEBUG main SpeBlockingClientImpl.request - error: org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://localhost:8700/audiofile/registration": localhost:8700 failed to respond; nested exception is org.apache.http.NoHttpResponseException: localhost:8700 failed to respond

jamesdbloom commented 5 years ago

This issue is fixed but likely it seems related to sockets staying open after Netty has reported they are closed and everything is shutdown. I have fixed this issue by adding a 2 second delay after this which seems to work reliably when tested. However I wouldn't recommend recreating a new instance of MockServer for each test because it is completely unnecessary and is much more expensive then just calling reset. If you just call reset (or clear) before each test and only start and stop the MockServer at the start and end of the suite your tests will run faster and this issue will completely go away.
Although MockServer is very fast at starting up as an analogy if you were testing a web application you wouldn't typically redeploy the web application between each test method how ever fast it was because that is overkill and as your test number increase the time taken to deploy would be significant.