search

mumble-voip / mumble

Mumble is an open-source, low-latency, high quality voice chat software.
https://www.mumble.info
Other
6.28k stars 1.11k forks source link

Public server registration not working due to SSL error #4569

Closed pb-programmer closed 3 years ago

pb-programmer commented 3 years ago

Quick disclaimer: I've been running my mumble server for several years and haven't touched it (except: apt upgrade) until I noticed this problem, so outdated configuration/regression is absolutely possible! I also upgraded my server to Ubuntu 20.04 LTS recently (started with 16.04 LTS, upgraded to 18.04 LTS in 2018, now switched to 20.04 LTS) and that might very well be the time when the bug occured first. I personally didn't notice the exact time because murmur works just fine as long as the server is in your "favorite" list.

Steps to Reproduce On Ubuntu 20.04.01 LTS with the stable PPA version of mumble (1.3.3-1\~ppa1\~focal1) installed and OpenSSL 1.1.1f Configuration file:

After startup the murmur log looks like this:

<W>2020-11-05 12:40:20.790 SSL: OpenSSL version is 'OpenSSL 1.1.1f  31 Mar 2020'
<W>2020-11-05 12:40:20.802 Initializing settings from /etc/mumble-server.ini (basepath /etc)
<W>2020-11-05 12:40:23.901 MetaParams: TLS cipher preference is "TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AE
S128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:AES256-SHA:AES128-SHA"
<W>2020-11-05 12:40:24.213 ServerDB: Opened SQLite database /var/lib/mumble-server/mumble-server.sqlite
<W>2020-11-05 12:40:24.215 ServerDB: Using SQLite's default rollback journal.
<W>2020-11-05 12:40:24.243 DBus registration succeeded
<W>2020-11-05 12:40:24.272 MurmurIce: Endpoint "tcp -h 127.0.0.1 -p 6502 -t 60000" running
<W>2020-11-05 12:40:24.727 Murmur 1.3.3 (1.3.3-1~ppa1~focal1) running on X11: Ubuntu 20.04.1 LTS: Booting servers
<W>2020-11-05 12:40:24.763 1 => Server listening on 0.0.0.0:64738
<W>2020-11-05 12:40:26.066 1 => Announcing server via bonjour
[...]
<W>2020-11-05 12:41:58.285 1 => <2:(-1)> New connection: 52.201.18.254:41916
<W>2020-11-05 12:41:58.290 1 => <2:(-1)> Connection closed: Error during SSL handshake: error:14209102:SSL routines:tls_early_post_process_client_hello:unsupported protocol [13]
<W>2020-11-05 12:41:58.487 1 => <3:(-1)> New connection: 52.201.18.254:41918
<W>2020-11-05 12:41:58.500 1 => <3:(-1)> Connection closed: Error during SSL handshake: error:14209102:SSL routines:tls_early_post_process_client_hello:unsupported protocol [13]
<W>2020-11-05 12:41:58.604 1 => Registration: DNS name callback mumble.christoph-mayer.net=>82.165.163.205 failed: Source-based callback failed. Server not reachable.

Noticed problems The DNS callback problem is weird. The domain name and the IP are correct, haven't changed in years and can be resolved just fine (checked several DNS servers all over the world). Another quick test showed the server is reachable with Mumble 1.3.3 (Win10), can be pinged and connected to via openssl. So I think this is just because the registration server can't connect.

Which brings us to real problem: The registration server can't connect

Registration used to work just fine a couple months ago (I don't look into the logs that often) and as a user mumble/mumur just work as intended (except the lack of "public server"-listing).

Further investigation I started digging around and noticed: If I connect to my server via openssl with standard settings it works just fine (successful connection): openssl s_client -connect mumble.christoph-mayer.net:64738 resulting murmur log:

<W>2020-11-05 12:51:05.415 1 => <4:(-1)> New connection: 46.142.168.###:52999
<W>2020-11-05 12:51:12.320 1 => <4:(-1)> Connection closed: The remote host closed the connection [1]

But if I connect to my server via openssl and force TLS1.0, the connection is immediately closed and I get the same error message the registration server produces openssl s_client -connect mumble.christoph-mayer.net:64738 -tls1 resulting murmur log:

<W>2020-11-05 12:51:22.016 1 => <5:(-1)> New connection: 46.142.168.###:53001
<W>2020-11-05 12:51:22.022 1 => <5:(-1)> Connection closed: Error during SSL handshake: error:14209102:SSL routines:tls_early_post_process_client_hello:unsupported protocol [13]
<W>2020-11-05 12:51:22.959 Ignored OpenSSL error 140E0197 for 0x55b09f2da5b0

So the registration server probably only tries to connect with TLS 1.0. Is there any way to allow murmur under Ubuntu 20.04 LTS to accept TLS 1.0 connections or is it possible to upgrade the registration server to offer newer TLS versions?

Krzmbrzl commented 3 years ago

The registration server can't connect

What do you mean by that?

I think I still didn't understand when this error occurs. Is someone actually trying to connect to your server (a client) or does the error appear in the log simply after having started the server? And is the server still visible in the public server list?

Popkornium18 commented 3 years ago

I also couldn't get registration to work and I have configured the cipherlist. After reading this bug report I tried it with the default cipherlist and sure enough: Registration: New registration successful

So I can confirm this.

Krzmbrzl commented 3 years ago

Not a crypto-expert here, but just to be sure: The problem is not the cipher-list but the handling of a custom list? Aka: Does it matter which ciphers you actually list there?

Popkornium18 commented 3 years ago

I think it's just a cipher mismatch when the murmur server tries to contact the registration server with a limited cipher list. I was using sslCiphers=EECDH+AESGCM:EDH+AESGCM and the registration server does not support these (I guess - All these cipher names can get really confusing).

The registration server also seems to be lacking TLS1.3 support. https://www.ssllabs.com/ssltest/analyze.html?d=publist-registration.mumble.info

pb-programmer commented 3 years ago

@Krzmbrzl I start murmur, it tries to register itself as a public server but when the registration server tries to connect to my server the connection is closed. So my server does not appear on the public server list.

If you know the url/ip everything works just fine. The log I showed appears all the time. It's the mumble registration server trying to connect to my server (but failing) to verify it's live.

@Popkornium18 Okay, I out commented everything cipher related from my mumble-server.ini and restarted murmurd. But sadly the same error still occurs

<W>2020-11-05 17:25:43.074 SSL: OpenSSL version is 'OpenSSL 1.1.1f  31 Mar 2020'
<W>2020-11-05 17:25:43.074 Initializing settings from /etc/mumble-server.ini (basepath /etc)
<W>2020-11-05 17:25:43.609 MetaParams: TLS cipher preference is "TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:AES256-SHA:AES128-SHA"
<W>2020-11-05 17:25:43.648 ServerDB: Opened SQLite database /var/lib/mumble-server/mumble-server.sqlite
<W>2020-11-05 17:25:43.649 ServerDB: Using SQLite's default rollback journal.
<W>2020-11-05 17:25:43.660 DBus registration succeeded
<W>2020-11-05 17:25:43.661 MurmurIce: Endpoint "tcp -h 127.0.0.1 -p 6502 -t 60000" running
<W>2020-11-05 17:25:43.721 Murmur 1.3.3 (1.3.3-1~ppa1~focal1) running on X11: Ubuntu 20.04.1 LTS: Booting servers
<W>2020-11-05 17:25:43.737 1 => Server listening on 0.0.0.0:64738
<W>2020-11-05 17:25:44.287 1 => Announcing server via bonjour
<W>2020-11-05 17:25:46.134 1 => <1:(-1)> New connection: 46.142.168.###:57410
<W>2020-11-05 17:25:46.237 1 => <1:(-1)> Client version 1.3.3 (Win: 1.3.3)
<W>2020-11-05 17:25:46.246 1 => Starting voice thread
<W>2020-11-05 17:25:46.253 1 => CELT codec switch ffffffff80000010 0 (prefer ffffffff80000010) (Opus 1)
<W>2020-11-05 17:25:46.263 1 => <1:###(3)> Authenticated
<W>2020-11-05 17:28:02.964 1 => <2:(-1)> New connection: 52.201.18.254:34284
<W>2020-11-05 17:28:02.973 1 => <2:(-1)> Connection closed: Error during SSL handshake: error:14209102:SSL routines:tls_early_post_process_client_hello:unsupported protocol [13]
<W>2020-11-05 17:28:03.174 1 => <3:(-1)> New connection: 52.201.18.254:34286
<W>2020-11-05 17:28:03.180 1 => <3:(-1)> Connection closed: Error during SSL handshake: error:14209102:SSL routines:tls_early_post_process_client_hello:unsupported protocol [13]
<W>2020-11-05 17:28:03.282 1 => Registration: DNS name callback mumble.christoph-mayer.net=>82.165.163.205 failed: Source-based callback failed. Server not reachable.
<W>2020-11-05 17:28:06.097 Ignored OpenSSL error 140E0197 for 0x558e73ac7060

My config is attached so you can check I didn't make a stupid mistake in there mumble-server.txt

Popkornium18 commented 3 years ago

So if I comment out sslCiphers murmur uses ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:AES256-SHA:AES128-SHA as the default for me, which is different than your default, which makes sense, since you use Ubuntu 20.04 and I use Debian 10.

Are you able to register if you explicitly set the ciphers that work for me?

pb-programmer commented 3 years ago

@Popkornium18 I'm not good with SSL cipher names and what to use where, so correct me if I'm wrong! I set my mumble-server.ini to: sslCiphers=ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:AES256-SHA:AES128-SHA

If I put that same string into "openssl cipher $YOURLIST" I get: TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:AES256-SHA:AES128-SHA

Which matches what murmur prints in it's log: <W>2020-11-05 18:02:19.987 MetaParams: TLS cipher preference is "TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:AES256-SHA:AES128-SHA"

Sadly the registration server still can't connect:

<W>2020-11-05 18:05:14.178 1 => <2:(-1)> New connection: 52.201.18.254:37152
<W>2020-11-05 18:05:14.183 1 => <2:(-1)> Connection closed: Error during SSL handshake: error:14209102:SSL routines:tls_early_post_process_client_hello:unsupported protocol [13]
<W>2020-11-05 18:05:14.379 1 => <3:(-1)> New connection: 52.201.18.254:37154
<W>2020-11-05 18:05:14.384 1 => <3:(-1)> Connection closed: Error during SSL handshake: error:14209102:SSL routines:tls_early_post_process_client_hello:unsupported protocol [13]
<W>2020-11-05 18:05:14.485 1 => Registration: DNS name callback mumble.christoph-mayer.net=>82.165.163.205 failed: Source-based callback failed. Server not reachable.
<W>2020-11-05 18:05:16.117 Ignored OpenSSL error 140E0197 for 0x562477b4a7a0

EDIT: Also tried sslCiphers=HIGH:MEDIUM and ended up with this behemoth in the log: <W>2020-11-05 18:26:55.192 MetaParams: TLS cipher preference is "TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES256-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:RSA-PSK-AES256-GCM-SHA384:DHE-PSK-AES256-GCM-SHA384:RSA-PSK-CHACHA20-POLY1305:DHE-PSK-CHACHA20-POLY1305:ECDHE-PSK-CHACHA20-POLY1305:AES256-GCM-SHA384:PSK-AES256-GCM-SHA384:PSK-CHACHA20-POLY1305:RSA-PSK-AES128-GCM-SHA256:DHE-PSK-AES128-GCM-SHA256:AES128-GCM-SHA256:PSK-AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:ECDHE-PSK-AES256-CBC-SHA384:ECDHE-PSK-AES256-CBC-SHA:SRP-RSA-AES-256-CBC-SHA:SRP-AES-256-CBC-SHA:RSA-PSK-AES256-CBC-SHA384:DHE-PSK-AES256-CBC-SHA384:RSA-PSK-AES256-CBC-SHA:DHE-PSK-AES256-CBC-SHA:AES256-SHA:PSK-AES256-CBC-SHA384:PSK-AES256-CBC-SHA:ECDHE-PSK-AES128-CBC-SHA256:ECDHE-PSK-AES128-CBC-SHA:SRP-RSA-AES-128-CBC-SHA:SRP-AES-128-CBC-SHA:RSA-PSK-AES128-CBC-SHA256:DHE-PSK-AES128-CBC-SHA256:RSA-PSK-AES128-CBC-SHA:DHE-PSK-AES128-CBC-SHA:AES128-SHA:PSK-AES128-CBC-SHA256:PSK-AES128-CBC-SHA"

Sadly the registration still does not work, so I think it's a TLS version mismatch, not a cipher mismatch

Krzmbrzl commented 3 years ago

The public server registration implementation is ancient. Thus it is definitely possible that it is just outdated

davidebeatrici commented 3 years ago

We received quite a few reports about this, it's almost certainly due to OpenSSL not being up-to-date on our server.

pb-programmer commented 3 years ago

Just as a quick follow up: With Ubuntu 20.04 LTS all system TLS libraries (OpenSSL, GnuTLS, NSS) have been set to default to security level 2, which in turn means TLS 1.2 or TLS 1.3 is enforced (+corresponding ciphers) for secure connections. TLS 1.0, TLS 1.1 or any SSL connections are instantly dropped. See here: https://discourse.ubuntu.com/t/default-to-tls-v1-2-in-all-tls-libraries-in-20-04-lts/12464 Apparently there has also been a blog post about this (from May 20th 2020), but it has since been withdrawn and you can't access it anymore.

Possible workaround

I personally think updating the public registration server would be the way to go, but I don't know if or when this is going to happen. Since a public mumble server without public registration is kind of pointless, I've got a workaround for anyone using murmur on Ubuntu 20.04 LTS for the time being.

This is NOT recommended, do this at your own risk and only if you are sure you fully understand all the drawbacks and possible security vulnerability this change brings!

In your mumble-server.ini change the cipher line to: sslCiphers=HIGH:MEDIUM:@SECLEVEL=1 Follow up with a sudo systemctl daemon-reload and/or a sudo systemctl restart mumble-server and you should be good to go.

You can verify the change worked when with openssl s_client -connect localhost:64738 -cipher "MEDIUM:HIGH:@SECLEVEL=1" -tls1 a connection is established and stays open. Or just wait a couple of minutes and look if there has been a (...) => Registration: Registration updated in your mumble-server.log

davidebeatrici commented 3 years ago

Our server now runs Ubuntu 20.04.

Please let us whether the issue is fixed.

pb-programmer commented 3 years ago

Wow that was fast, thanks a lot. I reverted my sslCipher back to default (from mumble wiki) sslCiphers=EECDH+AESGCM:EDH+aRSA+AESGCM:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:AES256-SHA:AES128-SHA

Looks like the public registration server can successfully connect to murmur now, sadly there still seems to be something wrong:

<W>2020-11-15 13:13:30.678 1 => <1:(-1)> New connection: 52.201.18.254:33016
<W>2020-11-15 13:13:30.684 1 => <1:(-1)> Connection closed: The remote host closed the connection [1]
<W>2020-11-15 13:13:30.751 1 => <2:(-1)> New connection: 52.201.18.254:33018
<W>2020-11-15 13:13:30.756 1 => Registration: DNS name callback mumble.christoph-mayer.net=>82.165.163.205 failed: Source-based callback failed. Server not reachable.
<W>2020-11-15 13:13:30.760 1 => <2:(-1)> Connection closed: The remote host closed the connection [1]

This seems like a separate issue though, so I'll investigate a bit to make sure the fault is not on my side...

Popkornium18 commented 3 years ago

Same for me

davidebeatrici commented 3 years ago

Thank you for your report!

Two different issues have just been fixed:

  1. Registration script attempting to connect using TLSv1 for Murmur 1.2.0+ and SSLv3 for older versions. The logic is now completely removed and we let OpenSSL negotiate the best cipher. The minimum TLS version is 1.2.
  2. No IPv6 support due to publist-registration.mumble.info being an alias to the VPS' IPv4 hostname. An AAAA record has been added to mumble.info and publist-registration.mumble.info is now an alias to mumble.info.

The last issue appears to be #4184, which we will fix as soon as possible

pb-programmer commented 3 years ago

Thank you very much for addressing the issue. I just checked my logs and without any further changes to the config and without a restart registration is now working as expected.

<W>2020-11-16 04:46:42.111 1 => <38:(-1)> New connection: 52.201.18.254:54176
<W>2020-11-16 04:46:42.116 1 => <38:(-1)> Connection closed: The remote host closed the connection [1]
<W>2020-11-16 04:46:42.204 1 => Registration: DNS name callback mumble.christoph-mayer.net=>82.165.163.205 failed: Source-based callback failed. Server not reachable.
[...]
<W>2020-11-16 07:50:18.123 1 => <41:(-1)> New connection: 52.201.18.254:37772
<W>2020-11-16 07:50:18.225 1 => <41:(-1)> Connection closed: The TLS/SSL connection has been closed [1]
<W>2020-11-16 07:50:18.230 1 => Registration: Registration updated
davidebeatrici commented 3 years ago

No problem, thank you very much for testing!

inf0maniac commented 1 year ago

Hello there guys. I believe i am facin an issue related to the bug described here.

I am running mumble in docker (https://github.com/mumble-voip/mumble-docker). I want to have my server listed in the public registry as I did before some years ago with no hassle. In this opportunity, having all the fields related to public server registration filled in the murmur.ini file, i got this error in the verbose output:

2022-09-23 00:55:14.416 Murmur 1.4.287 running on Linux: Ubuntu 20.04.5 LTS [x64]: Booting servers 2022-09-23 00:55:14.457 1 => Server listening on 0.0.0.0:64738 2022-09-23 00:58:03.087 1 => <0:(-1)> New connection: 159.100.252.167:36672 2022-09-23 00:58:03.325 1 => Registration failed: Host requires authentication 2022-09-23 00:58:03.514 1 => <0:(-1)> Connection closed: The TLS/SSL connection has been closed [1] Believing this was merely an error of the docker version, i installed in my host the mumble-server using the official ppa (sudo add-apt-repository ppa:mumble/release) but i got exactly the same registration error. I have googled for a solution with no avail. I would appreciate any help on this issue. thank you
davidebeatrici commented 1 year ago

Did you set your registerHostname correctly?

inf0maniac commented 1 year ago

Did you set your registerHostname correctly?

yes, i added, and i also tried the suggested sslCiphers but it does not work, always get this error: Registration failed: Host requires authentication <0:(-1)> Connection closed: The TLS/SSL connection has been closed [1]