WSS over explicit proxies not implemented for asyncio

[francisATgwn]

Steps to Reproduce

1. Use asyncio
2. Use the Component API
3. Pass a proxy in the transport for a wss router
4. Invoke sample code thusly: sample.py wss://myrouter.com/ws realm1 myproxy.com:3128

Expected Result

autobahn connects successfully

Actual Result

2019-09-27T14:10:02 connecting once using transport type "websocket" over endpoint "tcp"
2019-09-27T14:10:02 Exception in callback WebSocketAdapterProtocol._consume.<locals>.process(<Future finished result=None>) at /home/fferrell/src/proxy_example/venv/lib/python3.6/site-packages/autobahn/asyncio/websocket.py:106
handle: <Handle WebSocketAdapterProtocol._consume.<locals>.process(<Future finished result=None>) at /home/fferrell/src/proxy_example/venv/lib/python3.6/site-packages/autobahn/asyncio/websocket.py:106>
Traceback (most recent call last):
  File "/usr/lib/python3.6/asyncio/events.py", line 145, in _run
    self._callback(*self._args)
  File "/home/fferrell/src/proxy_example/venv/lib/python3.6/site-packages/autobahn/asyncio/websocket.py", line 110, in process
    self._dataReceived(data)
  File "/home/fferrell/src/proxy_example/venv/lib/python3.6/site-packages/autobahn/websocket/protocol.py", line 1213, in _dataReceived
    self.consumeData()
  File "/home/fferrell/src/proxy_example/venv/lib/python3.6/site-packages/autobahn/websocket/protocol.py", line 1232, in consumeData
    self.processProxyConnect()
  File "/home/fferrell/src/proxy_example/venv/lib/python3.6/site-packages/autobahn/websocket/protocol.py", line 3514, in processProxyConnect
    self.startTLS()
  File "/home/fferrell/src/proxy_example/venv/lib/python3.6/site-packages/autobahn/asyncio/websocket.py", line 230, in startTLS
    raise Exception("WSS over explicit proxies not implemented")
Exception: WSS over explicit proxies not implemented

Sample Code

import sys
from autobahn.asyncio.component import Component, run

proxy_host, proxy_port = sys.argv[3].split( ':' )

comp = Component(
    transports = dict(
        url = sys.argv[1],
        proxy = dict(
            host = proxy_host,
            port = int( proxy_port )
        ),
    ),
    realm = sys.argv[2],
)

@comp.on_join
def joined( session, details ):
    print( 'session ready' )

run( [ comp ] )

Notes

This works with twisted. Change the import in the sample code to autobahn.twisted.component and it connects as expected.

[francisATgwn]

We have a hard dependency on using a proxy. Due to unknown reasons (https://github.com/pyca/cryptography/issues/7488#) we are moving away from twisted to asyncio.

Any pointers on reference implementations we could use to attempt implementing https://github.com/crossbario/autobahn-python/blob/master/autobahn/asyncio/websocket.py#L227?

[meejah]

Have you tried contacting Twisted? I'm sure they would like to fix any TLS bug (if this is indeed a Twisted bug).

[francisATgwn]

@meejah thanks for the quick reply

It is on my todo list to open an issue with the Twisted project on that (assuming you're referring to the "unregistered schema" error in the issue I linked, not this issue). We're migrating to asyncio for multiple reasons, had it on our todo list for years and the sudden regression in our app due to the linked issue is the forcing function. I'd like to take the time to craft a minimal reproduction of that issue with twisted first, but I've never used Twisted to make a websocket connection before, so have to get up to speed on that first.

Regarding this issue, we are running our app on asyncio and we are under an SLA to deliver the upgrade to its requirements.txt. I do not have the luxury of going back to twisted to get my job done right now.

[meejah]

Okay. I assume you could repeat with any "wrapped" TLS protocol that uses STARTTLS in Twisted (I very much doubt it's websocket-specific, since you haven't really gotten that far .. you're still doing "proxy stuff" when the error hits, I assume).

I don't know how asyncio implements this flow ("upgrade to TLS on a non-TLS connection"). If you're switching for other reasons, that's fine (I responded because IMO it's likely less work to fix the Twisted bug than implement the missing asyncio functionality).

I don't remember why Autobahn doesn't have corresponding asyncio proxy support (but it could be that asyncio itself doesn't have a good way to do this).

As far as reference code, the Twisted implementation is as good as you'll get (possibly including the Twisted start-tls codepaths). TLS-over-proxies gets weird ;)

[francisATgwn]

Duly noted. That is helpful info.

[oberstet]

I don't remember why Autobahn doesn't have corresponding asyncio proxy support (but it could be that asyncio itself doesn't have a good way to do this).

yeah, also forgot;) maybe historic reasons that are no longer true.

and: I agree TLS with proxies etc provides potential for lots of fun.

even more so: it depends on the specific kind of proxy, and with TLS, that might be a real intercepting middlebox, unwrapping your TLS, doing its "content control", and then recipher using a different key. And faking the client side transport in reverse.

such middleboxes usually only work with more client side stuff, like e.g. a browser with the proxy CA cert being added as a new root CA, so that the proxy can lie to you.

my approach: this is bad shit that must be avoided. and if there is a middlebox, I want to know, because I then must switch network. so I want my websocket connection to fail on such fancy intercepting middleboxes.

non-intercepting TLS middleboxes can't fuzz with TLS, and hence are more like a L4 proxy, a dumb packet forwarder ...