search

Matthias247 / jawampa

Web Application Messaging Protocol (WAMP v2) support for Java
Apache License 2.0
148 stars 57 forks source link

Does it support connection through a proxy server? #3

Open KMax opened 9 years ago

KMax commented 9 years ago

Does the library support connection through a proxy server? I want to run the library behind a proxy server to connect to a WAMP router, but it hangs on the connecting phase...Thank you in advance!

BTW: the http_proxy environment variable is set.

Matthias247 commented 9 years ago

Sorry, it's not yet supported. The support for proxy servers also landed only quite recently in the Netty 4.1 branch and is not yet available in an official Netty release. I think as soon as it is officially released I can try integrating it into jawampa. Copying the code for proxy handling directly into jawampa would also be an option, but I would prefer on relying on Netty for that.

KMax commented 9 years ago

I understand, but I need a workaround, could you show me how to handle proxy in jawampa? I would implement it in my fork.

Matthias247 commented 9 years ago

I quickly hacked something together by backporting some of Netty 4.1 proxy support to 4.0 and including that in the repo. You can try it out by using the proxy path. Use the new builder option withProxy(...) to configure it. I couldn't test it here without a proxy.

KMax commented 9 years ago

Thank you for quick reply!

Unfortunatelly, it doesn't work, I even switched to Netty 4.1.0.Beta4-SNAPSHOT to get proxy support implemented in https://github.com/netty/netty/pull/2817 (see my fork), but it also doesn't work.

I have no idea how to fix it, because I didn't work with Netty before that, I hope that they will fix it soon.

Matthias247 commented 9 years ago

I tried to debug it and found out that my backport to Netty 4.0 caused a nullpointerconnection which lead the connection to failing. I pushed an update to the proxy branch with which I can at least connect to a proxy and get a positive response from it. Couldn't test the complete WAMP connection as I have no remote server for it. Another issue is however the proxy itself. When I tried to connect to a normal ws://...:80/ws address (no SSL) I found out that my proxy won't forward/process the WebSocket handshake and let me switch to websocket mode. I will simply respond with some invalid page. It only seems to work with an SSL connection and a 443 port where the proxy will really provide a transparent connection which also can be used for websockets. This means if I use builder.withProxy(...).withSslContext(SslContext.newClientContext(InsecureTrustManagerFactory.INSTANCE)) I can successfully send out a handshake and get a response (but a 404 as my server is missing). But the exact behavior will depend on the proxy. You can put a breakpoint in ProxyHandler.channelRead an look at the content of the msg object to determine what kidn of packet the proxy sent you.

Then there's some kind of race condition left: Immediatly after the connection to the proxy is established the ProxyHandler will send out the proxy connect request but the websocket client handshaker will also send out the handshake request (without waiting for the proxy to signal success). This might work depending on the proxy or might not work. But changing that behavior takes a little more time :(

Matthias247 commented 9 years ago

Ok, I can successfully connect to the crossbar demo through a proxy with this configuration:

builder.witUri("wss://demo.crossbar.io/ws")
       .withRealm("crossbardemo")
       .withSslContext(SslContext.newClientContext(InsecureTrustManagerFactory.INSTANCE))
       .withProxy(new InetSocketAddress("...", ...), "", "")
KMax commented 9 years ago

@Matthias247, should I be able to connect to wss://demo.crossbar.io/ws server (without proxy server) too? I want to test my client to ensure that it works without proxy server, but for some reason I can't connect to it...This is my code:

WampClientBuilder builder = new WampClientBuilder();
builder.witUri("wss://demo.crossbar.io/ws")
           .withRealm("crossbardemo")
           .withSslContext(SslContext.newClientContext(InsecureTrustManagerFactory.INSTANCE))
           .withInfiniteReconnects()
           .withReconnectInterval(5, TimeUnit.SECONDS);
Matthias247 commented 9 years ago

@KMax Yes, you should also be able to connect to that server. It's the server that is used in the official crossbar demos (https://demo.crossbar.io/demo/pubsub/index.html).

KMax commented 9 years ago

Hmm...it's strange, I get SSLException: Received fatal alert: handshake_failure when I try to connect to wss://demo.crossbar.io/ws or to my WAMP router. Below you can see the stacktrace:

14:15:24.424 [main] DEBUG io.netty.handler.ssl.JdkSslContext - Default protocols (JDK): [TLSv1.2, TLSv1.1, TLSv1] 
14:15:24.424 [main] DEBUG io.netty.handler.ssl.JdkSslContext - Default cipher suites (JDK): [TLS_RSA_WITH_AES_128_GCM_SHA256, SSL_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_MD5, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA]
14:15:24.541 [main] DEBUG i.n.c.MultithreadEventLoopGroup - -Dio.netty.eventLoopThreads: 8
14:15:24.547 [main] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
14:15:24.547 [main] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
14:15:24.547 [main] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
14:15:24.548 [main] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: true
14:15:24.551 [main] DEBUG i.n.util.internal.PlatformDependent - UID: 1000
14:15:24.551 [main] DEBUG i.n.util.internal.PlatformDependent - Java version: 8
14:15:24.551 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noUnsafe: false
14:15:24.551 [main] DEBUG i.n.util.internal.PlatformDependent - sun.misc.Unsafe: available
14:15:24.552 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noJavassist: false
14:15:24.553 [main] DEBUG i.n.util.internal.PlatformDependent - Javassist: unavailable
14:15:24.553 [main] DEBUG i.n.util.internal.PlatformDependent - You don't have Javassist in your class path or you don't have enough permission to load dynamically generated classes.  Please check the configuration for better performance.
14:15:24.553 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.tmpdir: /tmp (java.io.tmpdir)
14:15:24.553 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
14:15:24.553 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false
14:15:24.580 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.noKeySetOptimization: false
14:15:24.580 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.selectorAutoRebuildThreshold: 512
14:15:24.639 [WampClientEventLoop] DEBUG r.i.a.d.s.m.WAMPMessagePublisher - Connecting to WAMP router [wss://lpmstreams.tk/ws]
14:15:24.668 [WampClientEventLoop] DEBUG i.n.util.internal.ThreadLocalRandom - -Dio.netty.initialSeedUniquifier: 0x5c6219ca855bea17 (took 2 ms)
14:15:24.691 [WampClientEventLoop] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.allocator.type: unpooled
14:15:24.691 [WampClientEventLoop] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.threadLocalDirectBufferSize: 65536
14:15:24.729 [WampClientEventLoop] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetectionLevel: simple
14:15:24.795 [WampClientEventLoop] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.maxCapacity.default: 262144
14:15:24.797 [WampClientEventLoop] DEBUG io.netty.util.internal.Cleaner0 - java.nio.ByteBuffer.cleaner(): available
14:15:24.801 [WampClientEventLoop] DEBUG i.n.h.c.h.w.WebSocketClientHandshaker13 - WebSocket version 13 client handshake key: Tw25batF4CfqV1FLhZHAYA==, expected response: EhdfDoCpabfNuZgNpthiyP+wLgA=
14:15:24.848 [WampClientEventLoop] DEBUG io.netty.handler.ssl.SslHandler - Failed to complete handshake
javax.net.ssl.SSLException: Received fatal alert: handshake_failure
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[na:1.8.0_25]
    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1646) ~[na:1.8.0_25]
    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1614) ~[na:1.8.0_25]
    at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1780) ~[na:1.8.0_25]
    at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1075) ~[na:1.8.0_25]
    at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:901) ~[na:1.8.0_25]
    at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:775) ~[na:1.8.0_25]
    at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[na:1.8.0_25]
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:995) [stream-publisher-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:921) [stream-publisher-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:867) [stream-publisher-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:247) [stream-publisher-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:147) [stream-publisher-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333) [stream-publisher-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319) [stream-publisher-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787) [stream-publisher-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:130) [stream-publisher-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) [stream-publisher-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) [stream-publisher-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) [stream-publisher-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) [stream-publisher-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
    at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116) [stream-publisher-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_25]
KMax commented 9 years ago

Solved it! The issue is described here. I just needed to install pyOpenSSL from sources as it said in Secure WebSocket and HTTPS tutorial.

Now I can connect to my WAMP router, but still I can't connect to wss://demo.crossbar.io/ws. I'll try to do the same through a proxy server.

Matthias247 commented 9 years ago

Glad you are achieving progress. I'm afraid I can't help you that much on the issues. From the log you have posted I'm not really sure whether you are facing an SSL or a websocket error. While some lines point to SSL there is also this one:

WebSocket version 13 client handshake key: Tw25batF4CfqV1FLhZHAYA==, expected response: EhdfDoCpabfNuZgNpthiyP+wLgA=

This points out that the websocket server does not respond correctly to our websocket handshake. In such cases Netty would close the connection.

For debugging whether the proxy let's you establish a tunneled connection I would proceed as written above: You can put a breakpoint in ProxyHandler.channelRead an look at the content of the msg object to determine what kind of packet the proxy sent you. Should be (Full)HttpResponse with StatusCode 200 or something like that.

KMax commented 9 years ago

So now everything works fine for one of my Java apps which is a simple CLI application, I even can use jawampa without the patch from proxy branch.

No, I'm testing SSL (without proxy for now) on a web application running on Wildfly 8.1.0.Final and for some reason it doesn't work. I get the following exception:

18:45:46,733 INFO  [stdout] (WampClientEventLoop) DEBUG io.netty.handler.ssl.SslHandler - Failed to complete handshake
18:45:46,733 INFO  [stdout] (WampClientEventLoop) java.nio.channels.ClosedChannelException: null

The full log:

18:45:45,924 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG io.netty.handler.ssl.JdkSslContext - Default protocols (JDK): [TLSv1.2, TLSv1.1, TLSv1] 
18:45:45,925 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG io.netty.handler.ssl.JdkSslContext - Default cipher suites (JDK): [TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, SSL_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_MD5, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA]
18:45:46,152 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG i.n.c.MultithreadEventLoopGroup - -Dio.netty.eventLoopThreads: 8
18:45:46,159 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
18:45:46,159 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
18:45:46,160 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
18:45:46,161 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: true
18:45:46,167 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG i.n.util.internal.PlatformDependent - UID: 1000
18:45:46,168 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG i.n.util.internal.PlatformDependent - Java version: 8
18:45:46,168 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noUnsafe: false
18:45:46,168 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG i.n.util.internal.PlatformDependent - sun.misc.Unsafe: available
18:45:46,169 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noJavassist: false
18:45:46,170 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG i.n.util.internal.PlatformDependent - Javassist: unavailable
18:45:46,170 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG i.n.util.internal.PlatformDependent - You don't have Javassist in your class path or you don't have enough permission to load dynamically generated classes.  Please check the configuration for better performance.
18:45:46,171 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.tmpdir: /tmp (java.io.tmpdir)
18:45:46,171 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
18:45:46,171 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false
18:45:46,189 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.noKeySetOptimization: false
18:45:46,189 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.selectorAutoRebuildThreshold: 512
18:45:46,242 INFO  [stdout] (ServerService Thread Pool -- 50) DEBUG r.i.a.d.a.s.WAMPMessagePublishingService - WAMP router [wss://lpmstreams.tk/ws] status: Disconnected
18:45:46,244 INFO  [stdout] (WampClientEventLoop) DEBUG r.i.a.d.a.s.WAMPMessagePublishingService - WAMP router [wss://lpmstreams.tk/ws] status: Connecting
18:45:46,488 INFO  [stdout] (WampClientEventLoop) DEBUG i.n.util.internal.ThreadLocalRandom - -Dio.netty.initialSeedUniquifier: 0x9361ddd8805af44c (took 1 ms)
18:45:46,513 INFO  [stdout] (WampClientEventLoop) DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.allocator.type: unpooled
18:45:46,514 INFO  [stdout] (WampClientEventLoop) DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.threadLocalDirectBufferSize: 65536
18:45:46,557 INFO  [stdout] (WampClientEventLoop) DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetectionLevel: simple
18:45:46,651 INFO  [stdout] (WampClientEventLoop) DEBUG io.netty.util.Recycler - -Dio.netty.recycler.maxCapacity.default: 262144
18:45:46,653 INFO  [stdout] (WampClientEventLoop) DEBUG io.netty.util.internal.Cleaner0 - java.nio.ByteBuffer.cleaner(): available
18:45:46,657 INFO  [stdout] (WampClientEventLoop) DEBUG i.n.h.c.h.w.WebSocketClientHandshaker13 - WebSocket version 13 client handshake key: NHKsylVZSep5JBo9fDoVKw==, expected response: ujJVYdo7RCI2aS24neOv7ZZV5Co=
18:45:46,725 INFO  [stdout] (WampClientEventLoop) DEBUG i.n.h.s.u.InsecureTrustManagerFactory - Accepting a server certificate: EMAILADDRESS=kolchinmax@gmail.com, CN=lpmstreams.tk, OU=ISST, O=ITMO University, L=Saint-Petersburg, ST=Saint-Petersburg, C=RU
18:45:46,733 INFO  [stdout] (WampClientEventLoop) DEBUG io.netty.handler.ssl.SslHandler - Failed to complete handshake
18:45:46,733 INFO  [stdout] (WampClientEventLoop) java.nio.channels.ClosedChannelException: null
18:45:46,735 INFO  [stdout] (WampClientEventLoop) DEBUG r.i.a.d.a.s.WAMPMessagePublishingService - WAMP router [wss://lpmstreams.tk/ws] status: Disconnected
trustin commented 9 years ago

I've just pushed a fix for HTTP proxy handler to 4.1. You might want to try it out and let me know if you don't need to fork anymore or there's still a problem.

Matthias247 commented 9 years ago

@trustin Thank you very much for informing me about the update! I guess its this commit? I will merge the changes.

Regarding the fork/branch thing: This is still needed because there is no official Netty 4.1 maven release which contains the proxy support. Therefore I needed to copy the proxy related files (and did a quick&dirty backport to 4.0). As this solutions was not that clean I didn't want to have it in the master branch. I will wait for 4.1 with proxy support there.

trustin commented 9 years ago

@Matthias247 Thanks for getting back to me. I hope to release 4.1 as soon as possible, although it's always time that I'm in short of ..

genadi-hp commented 9 years ago

Hi @Matthias247 , I am software engineer at HP and now we are working on POC for java to java communication using websockets.

We are trying to use your java wamp implementation and now we facing the problem with passing through proxy server. Going through this thread i know that current version (0.4.1) does not supports proxy traversing because Netty supports proxies only starting from 4.1.

Having this, i took netty 4.1.0-Beta7 version, jawampa 0.4.1 version and code from WampClientChannelFactoryResolver#getFactory method (from your proxy branch) but failed to pass the proxy - both ws and wss. In general i inherited NettyWampClientConnectorProvider class and in its createConnector method for IWampConnector changed initChannel method (just like you did in WampClientChannelFactoryResolver#getFactory method in proxy branch):

ChannelPipeline p = ch.pipeline(); SocketAddress proxyAddress = new InetSocketAddress(proxyHost, Integer.parseInt(proxyPort)); p.addLast(new HttpProxyHandler(proxyAddress)); if (sslCtx0 != null) { p.addLast(sslCtx0.newHandler(ch.alloc(), uri.getHost(), port)); } p.addLast(new HttpClientCodec(), new HttpObjectAggregator(8192), new WebSocketClientProtocolHandler(handshaker, false), new WebSocketFrameAggregator(MAX_WEBSOCKET_FRAME_SIZE), new WampClientWebsocketHandler(handshaker), connectionHandler); } }); .......

Client does not get handshake from server: with WireShark i can see that proxy gets some ACK from server but then nothing happens. If i terminate server then client understands that there is disconnection and client tries to reconnect (but obviously fails).

Can you please point me on what i am doing wrong? Or miss?

Thank you in advance, Genadi P.S. To exclude that the problem is with proxy itself (I used little proxy server https://github.com/adamfisk/LittleProxy) i tried tyrus websocket implementation and succeeded to pass the proxy with ws (did not check wss).

Matthias247 commented 9 years ago

Hi @genadi-hp thanks for your interest in jawampa. Unfortunatly I can't help you that much directly, because I currently don't have time to work on that topic personally. Your code for including the proxy into the pipeline looks ok for me. What I would do is put some brakepoints in the various netty handlers (proxy handler, websocket handshake, ...) to see where things don't go as expected. Probably Netty expect things different then your proxy. If it doesn't work out an alternative for you could be to build a connection provider based on Tyrus. The new architecture (which doesn't bind the jawampa core to Netty) makes that possible.

caillette commented 8 years ago

Hi @genadi-hp,

I found this issue when googling about an issue with Netty, WebSockets, and HTTP proxy. I'm posting my latest findings here in the hope one can find them useful.

First you should be aware that:

This is my lab setup:

This works:

This fails:

One day I'll try this:

After a lot of investigations at the wrong places, I discoved that Netty's HttpProxyHandler installs its own HttpClientCodec, which fools Netty's WebSocketClientHandshaker. Right after the handshake, the WebSocketClientHandshaker reorganizes the pipeline, and hits HttpProxyHandler's HttpClientCodec, instead of the one used for establishing the HTTP connection to be promoted to ws(s)://.

It seems that you hit more or less the same problem than me. Don't spend time using ws://. You could check if Netty's pipeline is in a correct state after WebSocket handshake.