mock-server / mockserver

MockServer enables easy mocking of any system you integrate with via HTTP or HTTPS with clients written in Java, JavaScript and Ruby. MockServer also includes a proxy that introspects all proxied traffic including encrypted SSL traffic and supports Port Forwarding, Web Proxying (i.e. HTTP proxy), HTTPS Tunneling Proxying (using HTTP CONNECT) and SOCKS Proxying (i.e. dynamic port forwarding).
http://mock-server.com
Apache License 2.0
4.57k stars 1.07k forks source link

Local Mockserver Client not able to establish HTTPS connection #800

Closed niteshamin closed 2 years ago

niteshamin commented 4 years ago

Describe the issue Unable to establish a HTTPS connection from a local MockServer client trying to connect to MockServer server instance running in kubernetes.

What you are trying to do Attempting to run a local instance of the MockServer client to run test locally while connecting to our system and MockServer server instance running on Kubernetes on AWS.

External interface to kubernetes needs to HTTPS. Our setup worked fine using HTTP connection.

MockServer version 5.10.0

To Reproduce Steps to reproduce the issue:

  1. Mockserver server instance is running on kubernetes

  2. Mockserver client instance is being attempted to be executed locally

  3. Code used:

        mockServerClient = new MockServerClient
        ("mockserver_on_kubernetes", 443)
        .withSecure(true);
    
        mockServerClient.reset();
  4. What error you saw

org.mockserver.client.SocketConnectionException: Channel handler removed before valid response has been received

at org.mockserver.client.HttpClientConnectionHandler.handlerRemoved(HttpClientConnectionHandler.java:19)
at io.netty.channel.AbstractChannelHandlerContext.callHandlerRemoved(AbstractChannelHandlerContext.java:979)
at io.netty.channel.DefaultChannelPipeline.callHandlerRemoved0(DefaultChannelPipeline.java:637)
at io.netty.channel.DefaultChannelPipeline.destroyDown(DefaultChannelPipeline.java:876)
at io.netty.channel.DefaultChannelPipeline.destroyUp(DefaultChannelPipeline.java:844)
at io.netty.channel.DefaultChannelPipeline.destroy(DefaultChannelPipeline.java:836)
at io.netty.channel.DefaultChannelPipeline.access$700(DefaultChannelPipeline.java:46)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelUnregistered(DefaultChannelPipeline.java:1392)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:198)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:184)
at io.netty.channel.DefaultChannelPipeline.fireChannelUnregistered(DefaultChannelPipeline.java:821)
at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:826)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at java.base/java.lang.Thread.run(Thread.java:830)

Expected behaviour HTTPS connection should be made with local MockServer client with Mockserver server running on Kubernetes

MockServer Log Request are not making it to MockServer server instance.

niteshamin commented 4 years ago

From the Wireshark trace, it appears DNS lookup is only being attempt locally.

image
niteshamin commented 4 years ago

when switching to HTTP, it does appear a local DNS lookup is attempted and then fails but then an external DNS lookup is made.

jamesdbloom commented 4 years ago

I'll look into this once I finished the current work on support OpenAPI and Swagger which should be in the next few days, hopefully.

jamesdbloom commented 4 years ago

I can't reproduce this issue. In addition as MockServer has no direct interaction with DNS it is extremely unlikely to be caused by MockServer.

To help you resolve this issue can you please provide more details about how you are deploying MockServer, for example are you using Docker for Desktop on MacOS which runs inside Hyperkit and so would be unreachable unless you specify a LoadBalancer service as follows:

helm upgrade --install --namespace mockserver --set service.type=LoadBalancer --set service.port=1080 mockserver http://www.mock-server.com/mockserver-5.10.0.tgz

I have updated the documentation to make this clearer in the new (currently unreleased) version, the documentation for the new unreleased version is already available here: https://5-11.mock-server.com/where/kubernetes.html#mockserver_url

niteshamin commented 4 years ago

Sorry, I believe I need to clarify that this issue we are experiencing is with the MockServer CLIENT part.

Specifically when I try to configure it for HTTPS via this code snippet below and running it locally on my MAC, we are NOT seeing a request coming out to the network. We confirmed this via a WireShark trace and working with our NET OPS team. Seems to fail after local DNS lookup per WireShark trace.

mockServerClient = new MockServerClient
("mockserver_on_kubernetes", 443)
.withSecure(true);
mockServerClient.reset();

When I configure for HTTP, we are NOT seeing this issue.

Let me know if can supply additional WireShark traces for comparison or other data points.

jamesdbloom commented 4 years ago

As I understand the DNS resolution in Netty in impacted by the /etc/resolv.conf in your instance of Linux and DNS is known to be problematic in Kubernetes, for example see here. https://github.com/netty/netty/issues/8880.

I can try and reproduce what you doing and see if I get the same issue.

jamesdbloom commented 2 years ago

Closing due to inactivity