mock-server / mockserver

MockServer enables easy mocking of any system you integrate with via HTTP or HTTPS with clients written in Java, JavaScript and Ruby. MockServer also includes a proxy that introspects all proxied traffic including encrypted SSL traffic and supports Port Forwarding, Web Proxying (i.e. HTTP proxy), HTTPS Tunneling Proxying (using HTTP CONNECT) and SOCKS Proxying (i.e. dynamic port forwarding).
http://mock-server.com
Apache License 2.0
4.57k stars 1.07k forks source link

Response base 64 encoded #912

Closed smainil closed 2 years ago

smainil commented 3 years ago

Describe the issue Response body is always base 64 encoded.

What you are trying to do When adding an expectation in Java with a body containing plain text or json, the body after the expectation creation is always base 64 encoded

MockServer version 5.11.1

To Reproduce Steps to reproduce the issue:

  1. How you are running MockServer (i.e maven plugin, docker, etc) Runningserver using maven plugin (result is same when running the server from command line)

  2. Code you used to create expectations

    new MockServerClient("localhost", 1080)
        .when(
            request()
                .withPath("/api/system/status")
                .withMethod("GET")
        )
        .respond(
            response()
                .withBody("{\"status\":\"UP\"}")
                .withHeaders(
                    new Header("content-type".toString(), "application/json; charset=utf-8")
                )
    
        )
  3. What error you saw No error but in the log we can see that the value is encoded

    11:01:24.586 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "  "httpResponse" : {[\r][\n]"
    11:01:24.586 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "    "statusCode" : 200,[\r][\n]"
    11:01:24.586 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "    "headers" : {[\r][\n]"
    11:01:24.586 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "      "content-type" : [ "application/json; charset=utf-8" ][\r][\n]"
    11:01:24.586 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "    },[\r][\n]"
    11:01:24.586 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "    "body" : "eyJzdGF0dXMiOiJVUCJ9"[\r][\n]"

Expected behaviour Response body not base 64 encoded but the exact value as the one we defined

MockServer Log

Loading JavaScript to validate ECMA262 regular expression in JsonSchema because java.util.regex package in Java does not match ECMA262
[INFO] 1080 creating expectation:

  {
    "id" : "fa5d0c31-39c8-4c05-a94a-49e53e0f38ad",
    "priority" : 0,
    "httpRequest" : {
      "method" : "GET",
      "path" : "/api/system/status"
    },
    "times" : {
      "unlimited" : true
    },
    "timeToLive" : {
      "unlimited" : true
    },
    "httpResponse" : {
      "statusCode" : 200,
      "headers" : {
        "content-type" : [ "application/json; charset=utf-8" ]
      },
      "body" : "eyJzdGF0dXMiOiJVUCJ9"
    }
  }

 with id:

  fa5d0c31-39c8-4c05-a94a-49e53e0f38ad

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/Users/09243m/.m2/repository/ch/qos/logback/logback-classic/1.0.13/logback-classic-1.0.13.jar!/org/slf4j/impl/StaticLogge
rBinder.class]
SLF4J: Found binding in [jar:file:/C:/Users/09243m/.m2/repository/org/mock-server/mockserver-netty/5.11.1/mockserver-netty-5.11.1-jar-with-dependencies.jar!/o
rg/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/Users/09243m/.m2/repository/org/slf4j/slf4j-simple/1.7.30/slf4j-simple-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.cla
ss]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
11:04:30.554 [main] DEBUG o.a.h.c.protocol.RequestAuthCache - Auth cache not set in the context
11:04:30.564 [main] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection request: [route: {}->http://localhost:1080][total kept alive: 0; route all
ocated: 0 of 2; total allocated: 0 of 20]
11:04:30.622 [main] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection leased: [id: 0][route: {}->http://localhost:1080][total kept alive: 0; rou
te allocated: 1 of 2; total allocated: 1 of 20]
11:04:30.631 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Opening connection {}->http://localhost:1080
11:04:30.641 [main] DEBUG o.a.h.c.HttpClientConnectionManager - Connecting to localhost/127.0.0.1:1080
11:04:30.646 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Executing request PUT /expectation HTTP/1.1
11:04:30.647 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Target auth state: UNCHALLENGED
11:04:30.648 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Proxy auth state: UNCHALLENGED
11:04:30.675 [main] DEBUG org.apache.http.headers - http-outgoing-0 >> PUT /expectation HTTP/1.1
11:04:30.675 [main] DEBUG org.apache.http.headers - http-outgoing-0 >> Content-Length: 369
11:04:30.676 [main] DEBUG org.apache.http.headers - http-outgoing-0 >> Content-Type: text/plain; charset=UTF-8
11:04:30.676 [main] DEBUG org.apache.http.headers - http-outgoing-0 >> Host: localhost:1080
11:04:30.676 [main] DEBUG org.apache.http.headers - http-outgoing-0 >> Connection: Keep-Alive
11:04:30.676 [main] DEBUG org.apache.http.headers - http-outgoing-0 >> User-Agent: Apache-HttpClient/4.3.2 (java 1.5)
11:04:30.676 [main] DEBUG org.apache.http.headers - http-outgoing-0 >> Accept-Encoding: gzip,deflate
11:04:30.677 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "PUT /expectation HTTP/1.1[\r][\n]"
11:04:30.677 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "Content-Length: 369[\r][\n]"
11:04:30.677 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "Content-Type: text/plain; charset=UTF-8[\r][\n]"
11:04:30.677 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "Host: localhost:1080[\r][\n]"
11:04:30.677 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
11:04:30.678 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "User-Agent: Apache-HttpClient/4.3.2 (java 1.5)[\r][\n]"
11:04:30.678 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
11:04:30.678 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "[\r][\n]"
11:04:30.678 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "{[\r][\n]"
11:04:30.678 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "  "httpRequest" : {[\r][\n]"
11:04:30.678 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "    "method" : "GET",[\r][\n]"
11:04:30.678 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "    "path" : "/api/system/status"[\r][\n]"
11:04:30.678 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "  },[\r][\n]"
11:04:30.679 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "  "httpResponse" : {[\r][\n]"
11:04:30.679 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "    "statusCode" : 200,[\r][\n]"
11:04:30.679 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "    "body" : "eyJzdGF0dXMiOiJVUCJ9",[\r][\n]"
11:04:30.679 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "    "headers" : [ {[\r][\n]"
11:04:30.679 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "      "name" : "content-type",[\r][\n]"
11:04:30.679 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "      "values" : [ "application/json; charset=utf-8" ][\r][\n]"
11:04:30.679 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "    } ][\r][\n]"
11:04:30.680 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "  },[\r][\n]"
11:04:30.680 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "  "times" : {[\r][\n]"
11:04:30.680 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "    "remainingTimes" : 0,[\r][\n]"
11:04:30.680 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "    "unlimited" : true[\r][\n]"
11:04:30.680 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "  }[\r][\n]"
11:04:30.682 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "}"
11:04:32.048 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "HTTP/1.1 201 Created[\r][\n]"
11:04:32.049 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "version: 5.11.1[\r][\n]"
11:04:32.049 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "deprecated: "/expectation" is deprecated use "/mockserver/expectation" instead[\r][\n]"
11:04:32.049 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "connection: keep-alive[\r][\n]"
11:04:32.049 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "content-type: application/json; charset=utf-8[\r][\n]"
11:04:32.049 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "content-length: 435[\r][\n]"
11:04:32.049 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "[\r][\n]"
11:04:32.049 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "[ {[\r][\n]"
11:04:32.050 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "  "id" : "fa5d0c31-39c8-4c05-a94a-49e53e0f38ad",[\r][\n]"
11:04:32.050 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "  "priority" : 0,[\r][\n]"
11:04:32.050 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "  "httpRequest" : {[\r][\n]"
11:04:32.050 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "    "method" : "GET",[\r][\n]"
11:04:32.050 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "    "path" : "/api/system/status"[\r][\n]"
11:04:32.050 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "  },[\r][\n]"
11:04:32.050 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "  "httpResponse" : {[\r][\n]"
11:04:32.050 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "    "statusCode" : 200,[\r][\n]"
11:04:32.051 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "    "headers" : {[\r][\n]"
11:04:32.051 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "      "content-type" : [ "application/json; charset=utf-8" ][\r][\n]"
11:04:32.051 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "    },[\r][\n]"
11:04:32.051 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "    "body" : "eyJzdGF0dXMiOiJVUCJ9"[\r][\n]"
11:04:32.051 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "  },[\r][\n]"
11:04:32.051 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "  "times" : {[\r][\n]"
11:04:32.051 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "    "unlimited" : true[\r][\n]"
11:04:32.051 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "  },[\r][\n]"
11:04:32.052 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "  "timeToLive" : {[\r][\n]"
11:04:32.052 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "    "unlimited" : true[\r][\n]"
11:04:32.052 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "  }[\r][\n]"
11:04:32.052 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "} ]"
11:04:32.061 [main] DEBUG org.apache.http.headers - http-outgoing-0 << HTTP/1.1 201 Created
11:04:32.061 [main] DEBUG org.apache.http.headers - http-outgoing-0 << version: 5.11.1
11:04:32.062 [main] DEBUG org.apache.http.headers - http-outgoing-0 << deprecated: "/expectation" is deprecated use "/mockserver/expectation" instead
11:04:32.062 [main] DEBUG org.apache.http.headers - http-outgoing-0 << connection: keep-alive
11:04:32.062 [main] DEBUG org.apache.http.headers - http-outgoing-0 << content-type: application/json; charset=utf-8
11:04:32.062 [main] DEBUG org.apache.http.headers - http-outgoing-0 << content-length: 435
11:04:32.071 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Connection can be kept alive indefinitely
11:04:32.083 [main] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection [id: 0][route: {}->http://localhost:1080] can be kept alive indefinitely
11:04:32.083 [main] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection released: [id: 0][route: {}->http://localhost:1080][total kept alive: 1; r
oute allocated: 1 of 2; total allocated: 1 of 20]
jamesdbloom commented 2 years ago

The response is not base64 encoded the logs you have provided don't show what response you are receiving.

The following code shows MockServer is working correct as expected:

ClientAndServer mockServer = ClientAndServer.startClientAndServer();
mockServer
    .when(
        request()
            .withPath("/api/system/status")
            .withMethod("GET")
    )
    .respond(
        response()
            .withBody("{\"status\":\"UP\"}")
            .withHeaders(
                new Header("content-type", "application/json; charset=utf-8")
            )

    );

CloseableHttpClient httpClient = HttpClientBuilder.create()
    .build();

HttpGet getRequest = new HttpGet("http://localhost:" + mockServer.getPort() + "/api/system/status");
CloseableHttpResponse resp = httpClient.execute(getRequest);

try (InputStreamReader isr = new InputStreamReader(resp.getEntity().getContent()); BufferedReader br = new BufferedReader(isr);) {
    String responseBody = br.lines().collect(Collectors.joining("\n"));
    System.out.println("responseBody = " + responseBody);
    assertThat(responseBody, is("{\"status\":\"UP\"}"));
}

mockServer.stop();

The logs output when that code is executed are as follows:

2022-02-09 12:49:16 5.11.3-SNAPSHOT INFO 56378 started on port: 56378 
2022-02-09 12:49:16 5.11.3-SNAPSHOT INFO 56378 creating expectation:

  {
    "httpRequest" : {
      "method" : "GET",
      "path" : "/api/system/status"
    },
    "httpResponse" : {
      "headers" : {
        "content-type" : [ "application/json; charset=utf-8" ]
      },
      "body" : "{\"status\":\"UP\"}"
    },
    "id" : "17089d51-fb7c-4c50-8b9c-7c0256206497",
    "priority" : 0,
    "timeToLive" : {
      "unlimited" : true
    },
    "times" : {
      "unlimited" : true
    }
  }

 with id:

  17089d51-fb7c-4c50-8b9c-7c0256206497

2022-02-09 12:49:16 5.11.3-SNAPSHOT INFO 56378 received request:

  {
    "method" : "GET",
    "path" : "/api/system/status",
    "headers" : {
      "content-length" : [ "0" ],
      "User-Agent" : [ "Apache-HttpClient/4.5.13 (Java/11.0.2)" ],
      "Host" : [ "localhost:56378" ],
      "Connection" : [ "Keep-Alive" ],
      "Accept-Encoding" : [ "gzip,deflate" ]
    },
    "keepAlive" : true,
    "secure" : false
  }

2022-02-09 12:49:16 5.11.3-SNAPSHOT INFO 56378 request:

  {
    "method" : "GET",
    "path" : "/api/system/status",
    "headers" : {
      "content-length" : [ "0" ],
      "User-Agent" : [ "Apache-HttpClient/4.5.13 (Java/11.0.2)" ],
      "Host" : [ "localhost:56378" ],
      "Connection" : [ "Keep-Alive" ],
      "Accept-Encoding" : [ "gzip,deflate" ]
    },
    "keepAlive" : true,
    "secure" : false
  }

 matched expectation:

  {
    "httpRequest" : {
      "method" : "GET",
      "path" : "/api/system/status"
    },
    "httpResponse" : {
      "headers" : {
        "content-type" : [ "application/json; charset=utf-8" ]
      },
      "body" : "{\"status\":\"UP\"}"
    },
    "id" : "17089d51-fb7c-4c50-8b9c-7c0256206497",
    "priority" : 0,
    "timeToLive" : {
      "unlimited" : true
    },
    "times" : {
      "unlimited" : true
    }
  }

2022-02-09 12:49:16 5.11.3-SNAPSHOT INFO 56378 returning response:

  {
    "headers" : {
      "content-type" : [ "application/json; charset=utf-8" ]
    },
    "body" : "{\"status\":\"UP\"}"
  }

 for request:

  {
    "method" : "GET",
    "path" : "/api/system/status",
    "headers" : {
      "content-length" : [ "0" ],
      "User-Agent" : [ "Apache-HttpClient/4.5.13 (Java/11.0.2)" ],
      "Host" : [ "localhost:56378" ],
      "Connection" : [ "Keep-Alive" ],
      "Accept-Encoding" : [ "gzip,deflate" ]
    },
    "keepAlive" : true,
    "secure" : false
  }

 for action:

  {
    "headers" : {
      "content-type" : [ "application/json; charset=utf-8" ]
    },
    "body" : "{\"status\":\"UP\"}"
  }

 from expectation:

  17089d51-fb7c-4c50-8b9c-7c0256206497

responseBody = {"status":"UP"}
2022-02-09 12:49:16 5.11.3-SNAPSHOT INFO 56378 stopped for port: 56378 
undernorthernsky commented 2 years ago

Probably stumbled upon this issue or something rather close. In my case it was the received request body which appeared base64 encoded when trying to check it in an ExpectationResponseCallback.

def makeRequest(suffix, body) {
    def post = new URL("http://localhost:5000/${suffix}").openConnection()
    post.setRequestMethod("POST")
    post.setDoOutput(true)
    post.setRequestProperty("Content-Type", "multipart/related")
    post.getOutputStream().write(body.getBytes("UTF-8"))
    post.getInputStream().getText()
  }

  def testFoo() {
    setup:
      mockServer.when(HttpRequest.request()
        .withPath("/foo")
      ).respond(
        HttpClassCallback.callback(VerifyFoo.name)
      )
    expect:
      'FOO' == makeRequest('foo', 'FOO')
  }

static class VerifyFoo implements ExpectationResponseCallback {
    @Override
    HttpResponse handle(HttpRequest httpRequest) throws Exception {
  // THIS FAILS
      String body = httpRequest.getBodyAsString()
 // THIS WORKS
      String body = new String(httpRequest.getBody().getRawBytes())
      println 'CHECK /foo: ' + body
      return HttpResponse.response()
          .withBody(body)
          .withStatusCode(200)
    }
  }

When the request Content-Type is set (and not "text/*") the content is mapped to a BinaryBody: https://github.com/mock-server/mockserver/blob/master/mockserver-core/src/main/java/org/mockserver/model/BinaryBody.java#L46