coaps+ws: SSL Context defaults to `None` which make websockets connect through HTTP

[jca-klk]

Hello!

I am experimenting with aiocoap and WebSockets over TLS (sheme coaps+ws://).

The basic client (/clientGET.py) fails when run with coaps+ws against the basic server (/server.py) configured behind a reverse-proxy. The server behaves correctly when I connect with a basic websockets module client.

A Wireshark capture showed that websockets attempts to connect to the server using clear-text HTTP, despite of the scheme. The TCP port, however, was correctly set to 443.

I dug down the issue to transports/ws.py _connect_task where ssl_context is set to None. Passing ssl=None to websockets.connect() makes it not create the TLS context with create_default_context().

websockets documentation on websockets.client.connect:

You can set ssl to a SSLContext to enforce TLS settings. When connecting to a wss:// URI, if ssl isn’t provided, a TLS context is created with create_default_context().

In this case, ssl is provided, but set to None by default.

It seems that the issue comes from credentials.py: CredentialsMap.ssl_client_context() which returns None when no credentials are configured, and expects the "user to fill in ssl.create_default_context()".

I believe aiocoap users should not have to configure any specific credentials when connecting to websockets over TLS, but rather have sane default as websockets module provides, i.e. ssl.create_default_context() somehow.

I am not sure how to fix this bug though, as I am not familiar with this library and how it handles Credentials.

Steps to reproduce

Server side

On a Linux machine with a public IP address and associated domain name, I ran the Caddy server as a TLS endpoint and reverse-proxy using the following Caddyfile. (Replace coap.example.com with the machine domain name). Caddy's reverse-proxy automatically handles websockets, and handles also TLS certificate request.

coap.example.com

reverse_proxy :8683

On the same server, I ran the example server server.py with no changes.

Client side

I ran the clientGET.py changing the Message() uri to coaps+ws://coap.example.com/time. I also enabled DEBUG logging.

It fails as follow (hostname redacted):

python clientGET.py
DEBUG:asyncio:Using selector: EpollSelector
DEBUG:websockets.client:= connection is CONNECTING
DEBUG:websockets.client:> GET /.well-known/coap HTTP/1.1
DEBUG:websockets.client:> Host: coap.example.com
DEBUG:websockets.client:> Upgrade: websocket
DEBUG:websockets.client:> Connection: Upgrade
DEBUG:websockets.client:> Sec-WebSocket-Key: 9VvdWi8weg9LjMFbtWb1Bw==
DEBUG:websockets.client:> Sec-WebSocket-Version: 13
DEBUG:websockets.client:> Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
DEBUG:websockets.client:> Sec-WebSocket-Protocol: coap
DEBUG:websockets.client:> User-Agent: Python/3.11 websockets/12.0
DEBUG:websockets.client:! failing connection with code 1006
DEBUG:websockets.client:x half-closing TCP connection
DEBUG:websockets.client:= connection is CLOSED
Failed to fetch resource:
did not receive a valid HTTP response

Wireshark capture of the request shows that something is wrong: "Dangerous misconfiguration".

Frame 4: 411 bytes on wire (3288 bits), 411 bytes captured (3288 bits)
Ethernet II, Src: HewlettP_XX:XX:XX (XX:XX:XX:XX:XX:XX), Dst: Redacted_YY:YY:YY (YY:YY:YY:YY:YY:YY)
Internet Protocol Version 4, Src: xxx.xxx.xxx.xxx, Dst: yyy.yyy.yyy.yyy
Transmission Control Protocol, Src Port: 38174, Dst Port: 443, Seq: 1, Ack: 1, Len: 345
Hypertext Transfer Protocol
    [Expert Info (Warning/Security): Unencrypted HTTP protocol detected over encrypted port, could indicate a dangerous misconfiguration.]
    GET /.well-known/coap HTTP/1.1\r\n
    Host: coap.example.com\r\n
    Upgrade: websocket\r\n
    Connection: Upgrade\r\n
    Sec-WebSocket-Key: 9VvdWi8weg9LjMFbtWb1Bw==\r\n
    Sec-WebSocket-Version: 13\r\n
    Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits\r\n
    Sec-WebSocket-Protocol: coap\r\n
    User-Agent: Python/3.11 websockets/12.0\r\n
    \r\n
    [Full request URI: http://coap.example.com/.well-known/coap]
    [HTTP request 1/1]
    [Response in frame: 7]

Resolution

Commenting out transports/ws.py line 286 altogether "fixes" the problem, but I guess there is a better solution.

Extra information

Please include the output of python3 -m aiocoap.cli.defaults in your report, run in the environment in which you experienced the issues.

Python version: 3.11.6 (main, Oct  8 2023, 05:06:43) [GCC 13.2.0]
aiocoap version: 0.4.7
Modules missing for subsystems:
    dtls: everything there
    oscore: everything there
    linkheader: everything there
    prettyprint: everything there
Python platform: linux
Default server transports:  oscore:tinydtls:tcpserver:tcpclient:tlsserver:tlsclient:ws:udp6
Selected server transports: oscore:tinydtls:tcpserver:tcpclient:tlsserver:tlsclient:ws:udp6
Default client transports:  oscore:tinydtls:tcpclient:tlsclient:ws:udp6
Selected client transports: oscore:tinydtls:tcpclient:tlsclient:ws:udp6
SO_REUSEPORT available (default, selected): True, True

[chrysn]

Thanks, I'm having a look.

On first glance, while this is definitely bad and needs to get fixed, I don't see how this would go unnoticed, because it'd need an equally broken server that takes unencrypted connections on an encrypted port, so I'm not treating this as security critical (the severity would be minimal, given it'd need an already compromised peer). If you disagree, please feel free to file for a CVE number for this, and report it here.

That websockets behaves subtly different from OpenSSL is regrettable. It should be straightforward to fix now that you have identified the issue, although it saddens me that I'll have to add the explicit code to use the system's CA storage (which is a choice I don't like for security reasons but which makes sense for practical reasons, and OpenSSL's defaults meant that I was deferring to their defaults rather than recommending system CAs myself -- anyhow, this needs fixing).

[jca-klk]

Hi! Thanks for the quick reply.

No, I do not think this is a security issue. The reverse-proxy replies with HTTP/1.0 400 Bad Request with data Client sent an HTTP request to an HTTPS server. It does not go through the proxy and the client closes the TCP connection right away.

My issue is indeed on the functional part, with default settings. I already encountered such issue with the websockets module, and I chose to only pass ssl parameter when required, using something like that:

wsargs = {}
if ssl_is_required:
    wsargs['ssl'] = my_ssl_context
ws = await websockets.connect(uri, **wsargs)

I am not sure it is easy to do in here. I feel I miss some context after reading commit 6270d9382a1589a7d228eb34a91607bdc625dc60

[chrysn]

Could you give #326 a try? It should fix this.

For context on the change that broke this: aiocoap can also run in the browser, in which case it can only use WebSockets connections, and they exit the browser through a websocket module that just calls out to the browser's JavaScript WebSocket API. That one is not implemented using the ssl module, and has no influence on which certificates are accepted.

Tests couldn't really catch the breakage because no certificates created for a test situation will be valid under the default (system-wide) context, unless the tests install their fake CA into the system root, which I'd consider excessive.

[jca-klk]

I confirm #326 works with my setup.

Thank you for the quick responses and fix. Kudos 🎉

[chrysn]

Indeed that should have been is not -- pyodide isn't quite easy to test, and I haven't gotten to running it so far.

Once I'm done with those tests, this will be merged.

Thanks for reporting this, and your quick feedback!

[chrysn]

Now tested on pyodide:

import micropip
await micropip.install("https://some-server-that-has-the-right-CORS-headers/aiocoap-0.4.7.post0-py3-none-any.whl")
from aiocoap import * 
ctx = await Context.create_client_context()
req = Message(code=GET, uri="coaps+ws://rd.coap.amsuess.com/.well-known/core")
await ctx.request(req).response