client: permission denied

[Stunkymonkey]

On the client connecting to the server on voice.stunkymonkey.de:40200:

[20:15:38] Welcome to Mumble.
[20:15:38] Connecting to server voice.stunkymonkey.de.
[20:15:38] Server connection failed: Permission denied.
[20:15:49] Reconnecting.
[20:15:49] Server connection failed: Permission denied.

server config:

database=user_mumble
dbDriver=QMYSQL
dbUsername=user
dbPassword=not4u2see
dbOpts="UNIX_SOCKET=/var/lib/mysql/mysql.sock"
welcometext="<br />Welcome to this server running <b>Murmur</b>.<br />Enjoy your stay!<br />"
port=40200
allowping=true
bonjour=false
sslCert=/home/user/etc/certificates/voice.stunkymonkey.de.crt
sslKey=/home/user/etc/certificates/voice.stunkymonkey.de.key

tried murmur server on centos 7: 1.3.0-rc1 & 1.3.0-rc2 and mumble client on arch linux: 1.3.0-rc1 & 1.3.0-rc2

i believe, that the client does not even try to connect.

[davidebeatrici]

Is there something related in the developer console?

[Stunkymonkey]

i searched for it on the internet and did not find a hit.

[Stunkymonkey]

how can i activate the developer console?

[davidebeatrici]

Configure -> Settings... -> User Interface -> Enable Developer menu

Once you've done that, you can access Developer Console from the Developer menu.

[Stunkymonkey]

<W>2019-07-01 21:14:13.106 QSqlDatabasePrivate::removeDatabase: connection 'ServerHandler' is still in use, all queries will cease to work.
<W>2019-07-01 21:14:13.107 QSqlDatabasePrivate::addDatabase: duplicate connection name 'ServerHandler', old connection removed.
<W>2019-07-01 21:14:13.129 Database SQLite: "3.28.0"
<W>2019-07-01 21:14:13.130 OpenSSL Support: 1 (OpenSSL 1.1.1c  28 May 2019)
<W>2019-07-01 21:14:13.173 ServerHandler: TLS cipher preference is "TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:AES256-SHA:AES128-SHA"
<W>2019-07-01 21:14:17.239 Non-toplevel surfaces can't request window states

[davidebeatrici]

I don't see any related issues.

What about the server log?

[Stunkymonkey]

[user@sanguin ~]$ /home/user/mumble/murmur.x86 -v -fg -ini /home/user/mumble/murmur.ini
<W>2019-07-01 21:23:26.776 SSL: OpenSSL version is 'OpenSSL 1.0.2k  26 Jan 2017'
<W>2019-07-01 21:23:26.777 Initializing settings from /home/user/mumble/murmur.ini (basepath /home/user/mumble)
<C>2019-07-01 21:23:26.779 MetaParams: Adding 1 intermediate certificates from certificate file.
<W>2019-07-01 21:23:26.873 MetaParams: TLS cipher preference is "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:AES256-SHA:AES128-SHA"
<W>2019-07-01 21:23:26.919 OSInfo: Failed to execute lsb_release
<W>2019-07-01 21:23:26.919 Murmur 1.3.0 (1.3.0-rc2) running on X11: Linux 3.10.0-957.21.3.el7.x86_64: Booting servers
<W>2019-07-01 21:23:26.944 1 => Server listening on 0.0.0.0:40200
<W>2019-07-01 21:23:27.046 1 => Not registering server as public

if i try to connect nothing is being shown with the client and server version 1.2.9 my config works

[Stunkymonkey]

was an arch build issue is now fixed

[davidebeatrici]

Do you know details about what the issue was?

[Stunkymonkey]

sorry no. just installed the new update and it works. build config diff is: https://git.archlinux.org/svntogit/community.git/commit/trunk?h=packages/mumble&id=577cafa8445cbbbbf592471d0d809cb189a0444b do not really know why there was a problem

[davidebeatrici]

No problem.

[theAkito]

@davidebeatrici I noticed this issue the first time in December 2018. I get Connection refused when connecting from Android and Permission denied when connecting from Debian. The problem now is, that it only happens on any Android client as well as any desktop Linux client. Windows clients are fine and didn't have any issues.

[d3adb5]

Been getting this issue on Arch, package version 1.4.230-8. The link @Stunkymonkey posted doesn't work anymore, as the Arch Linux maintainers moved the repository around to archlinux/svntogit-community. The commit linked just removes jack as a dependency of Mumble.

I tried uninstalling the optional dependencies that were installed on my system: lib32-glibc and speech-dispatcher, to no avail. I'm looking at the output of the mumble command itself, and the developer console, but nothing seems related to the "Permission denied" error I'm getting.

I know this issue lies on the client side, since other people, with different versions of the Mumble client, are able to connect to my server just fine. Moreover, I think this has to do with connecting to servers for the first time. Here's a list of things I've been trying:

• Remove user directories that have to do with Mumble: .config/Mumble, .local/share/Mumble, .local/share/data/Mumble
• Change /etc/hosts to point the address of a server I can connect to to the IP address of the server I want. THIS WORKS, but isn't ideal.
• Downgrade Mumble to 1.3.4 or past revisions of 1.4.230.

The issue clearly lies outside Mumble itself, and it isn't catching whatever error is happening, only that there was an error.

Any help debugging this would be much appreciated. It's been driving me insane.

[Krzmbrzl]

You might want to try if you can establish a TCP and/or UDP connection using some of the plain command line tools (I think openssh ships some tools that can do that) and see if the issue persists there as well :thinking:

[d3adb5]

You might want to try if you can establish a TCP and/or UDP connection using some of the plain command line tools (I think openssh ships some tools that can do that) and see if the issue persists there as well thinking

Do you mean doing something like an SSH tunnel and using localhost when connecting? It does work, but as expected it'll give me a certificate error and I'm still unsure where the problem might lie. Any ideas?

[Krzmbrzl]

No - completely outside and unrelated to Mumble. Just check if it is possible to establish a regular TCP and/or UDP connection to your server to rule out any issues on that level

[d3adb5]

Took me quite a long time to finally respond, but...

No - completely outside and unrelated to Mumble. Just check if it is possible to establish a regular TCP and/or UDP connection to your server to rule out any issues on that level

I can make regular TCP/UDP connections to it: I can communicate with other services on the same VM through TCP and UDP ports just fine. This seems to be an issue with just Mumble, and I'm pretty confused about it. Still an ongoing thing, though I noticed after a few package updates --- on the client side --- connecting to a Mumble server takes a while, but goes through eventually.

Any other ideas? This still puzzles me. :(

[Krzmbrzl]

Do you also have this issue on other Mumble servers or only this particular one?

[d3adb5]

I have it on other servers, yeah. Any other server that wasn't already on my list (so all of them but two) when this problem started happening gives me the same error.

[Krzmbrzl]

Okay, then that pretty much rules out anything server-related :thinking: Unfortunately it seems that we are handed this unhelpful error message directly from Qt, so there's not much that we can do about this. If you were able to compile Mumble yourself, I could give you a patch to apply that might increase the logging verbosity a little bit so that we could try to figure this out...

[d3adb5]

I'd assumed it would've been a problem with one of Mumble's dependencies, but I really didn't think Qt might've been the culprit. Alright, send me the patch, I'll compile Mumble and get back to you with any results. Thank you so much for investing any time into this, I can gather it's a very specific, setup-related issue rather than one with Mumble itself.

[Krzmbrzl]

diff --git a/src/mumble/MainWindow.cpp b/src/mumble/MainWindow.cpp
index cec90ccbc..55e93f962 100644
--- a/src/mumble/MainWindow.cpp
+++ b/src/mumble/MainWindow.cpp
@@ -3401,7 +3401,7 @@ void MainWindow::serverDisconnected(QAbstractSocket::SocketError err, QString re

                if (!reason.isEmpty()) {
                        Global::get().l->log(Log::ServerDisconnected,
-                                                                tr("Server connection failed: %1.").arg(reason.toHtmlEscaped()));
+                                                                tr("Server connection failed after having been connected: %1 (%2).").arg(reason.toHtmlEscaped()).arg(err));
                } else {
                        Global::get().l->log(Log::ServerDisconnected, tr("Disconnected from server."));
                }
@@ -3461,9 +3461,9 @@ void MainWindow::serverDisconnected(QAbstractSocket::SocketError err, QString re
        AudioInput::setMaxBandwidth(-1);
 }

-void MainWindow::resolverError(QAbstractSocket::SocketError, QString reason) {
+void MainWindow::resolverError(QAbstractSocket::SocketError error, QString reason) {
        if (!reason.isEmpty()) {
-               Global::get().l->log(Log::ServerDisconnected, tr("Server connection failed: %1.").arg(reason.toHtmlEscaped()));
+               Global::get().l->log(Log::ServerDisconnected, tr("Server connection failed: %1 (%2).").arg(reason.toHtmlEscaped()).arg(error));
        } else {
                Global::get().l->log(Log::ServerDisconnected, tr("Server connection failed."));
        }

but I really didn't think Qt might've been the culprit.

It probably isn't but as the code is currently written it seems that at least the error message that is handed down to us is a bit unhelpful. Thus, in my patch above I am trying to get Mumble to print the underlying error code which we can use to look up the underlying error (because interestingly enough upon investigation a permission denied error should not exist). Also the patch will help us to identify which code path is being taken when this error happens. Either you get the regular Server connection failed: ... message or Server connection failed after having been connected: ....

EDIT: This patch will apply to master at 2ca3edce8af4be62dd2373bafeb491aa533e671e

[d3adb5]

Once again I took a week to reply, but here I am with the result: [20:01:16] Server connection failed after having been connected: Permission denied (3).

Something unrelated, which I don't know if I mentioned: I've also been noticing the problem described in #3882. With a localhost server it shows it on the table and lets me connect to it immediately just fine, but while that was the behavior with remote servers before, now they take up to 10 seconds to display as connectable and another 10 seconds me to actually make the connection. Dunno if I mentioned it. Thank you for being patient and taking time to look into this, again.

[Krzmbrzl]

Hm... What port is your server using? The only reason why this kind of error could be emitted by Qt that I found online was that the used port was in some "ephimeral range" on WIndows, meaning that the respective port could already be occupied by a different application,,,

[d3adb5]

It's using the default port. It's running in a Docker container on a VPS, and I'm running the client on Arch Linux. Nothing else is using the 64738 port on the server side (I'm also able to connect to it if I mess with /etc/hosts and use a "known" domain, like I described) and not that many applications are connecting to the Internet on my PC. :(

[davidebeatrici]

I would check the dynamic port range, for reference: https://docs.microsoft.com/en-us/troubleshoot/windows-server/networking/default-dynamic-port-range-tcpip-chang

[d3adb5]

Neither server or client are running Windows, though. Like I said, this issue isn't even with only my server: any server I try to connect to, save for two that I'd connected to before this issue began, leads me to the same issue. I can circumvent it by messing with /etc/hosts:

I've connected to server abc.example.com before this issue began. If I go on /etc/hosts and point abc.example.com to another server's IP address, I am able to connect to the server through Mumble and ignore the SSL cert warning.

It's not just my server, and the issue is present on another machine of mine running Arch Linux. I'm just not sure how to go about diagnosing and solving this problem. :(

[davidebeatrici]

Sorry, on Linux you should check net.ipv4.ip_local_port_range.

I would also monitor the network traffic with WireShark.

[d3adb5]

I would also monitor the network traffic with WireShark.

I should've thought of that sooner. Thank you so much for the suggestion, @davidebeatrici! And thank you so much for your help, @Krzmbrzl. I really appreciate it. Here's what I found out, maybe you guys can help me figure out the Mumble issue:

1. I was able to establish a TCP connection to the VM the server is on, and even the server itself, except... when trying to establish an IPv6 connection to the Mumble server with telnet, I get a Permission denied error!
2. WireShark revealed Mumble is querying the DNS server for SRV records when it starts up, but despite the query failing, it somehow waits 10 seconds to query the DNS server for A and AAAA records.

I didn't know about the SRV record, but after adding it, I no longer have any delays connecting to my servers. I wonder what is causing the 10 second delay inbetween DNS queries. Do you guys have any guesses?

About the Permission denied situation: it has nothing to do with Mumble, and instead with firewalld and Docker. Seems I'm unable to connect to anything in the docker zone using IPv6, and disabling firewalld lets me connect to the server just fine. I'm so sorry for the trouble and confusion I caused. :frowning_face: