webrtc branch

[streaps]

I wonder what the plans for the webrtc branch are? (or if there are any). While master had a couple of commits recently, the last changes in webrtc are from February last year.

Do I understand it correctly that the browser's opus codec is used with the webrtc branch?

[Johni0702]

I wonder what the plans for the webrtc branch are?

Long term I'd like to merge the webrtc branch into master as I find the current mumble-web implementation fundamentally flawed (see the "Motivation" section of the thread linked below). I haven't been working on mumble-web in general for the last year, only been accepting PRs, hence why there have been changes on master but not on webrtc.

Do I understand it correctly that the browser's opus codec is used with the webrtc branch?

Yes, it delegates all its audio processing (except VAD) to the browser, which should make it far more resilient to performance issues. For technical details see: https://github.com/mumble-voip/mumble/issues/3561

[streaps]

Sounds promising :)

So I tried the webrtc branch again (the last time I gave up at some point). Compiling mumble-web-proxy was easy this time. On Alpine Linux 3.11 I had to install openssl-dev and glib-dev as additional requirements (libssl-dev and libglib2.0-dev on Debian).

With the mumble-web client I had the same problem as described in #42, which is not really surprising, because it's still the same version.

I would be nice if we could get the webrtc functionality into master. With Covid-19 and many people working remotely, mumble-web is more needed than ever before.

[streaps]

I merged master into the webrtc branch to see what happens. I'm still having the same issue. This is what I have in the ./node_modules/mumble-client/ directory. The date 1985 is weird.

-rw-r--r--    1 root     root          4087 Oct 26  1985 README.md
-rw-r--r--    1 root     root            52 Oct 26  1985 index.js
drwxr-xr-x    6 root     root             6 Apr 20 08:46 node_modules
-rw-r--r--    1 root     root          2107 Apr 20 08:46 package.json

[Johni0702]

I just properly merged master into webrtc, it should build now and have all the same features (well, except the CELT codec but it's rather old and there's no reasonable way around that). Even deployed a demo instance at: https://voice.johni0702.de/webrtc/?address=voice.johni0702.de&port=443/demo The websockify running for the demo instance has actually been mumble-web-proxy for quite some time now, so at least the non-webrtc part of it should be fairly solid. Should you run into any issue with it which isn't present in the non-webrtc version, please don't hesitate to tell me.

I also added a note to the master README nudging users into trying the webrtc branch. That way it should get some more testing and can eventually become the main version. Though before that happens, I'd like the protocol changes required for it to become official, so they don't run the (rather high) risk of being incompatible with newer Mumble versions.

[streaps]

The webrtc client I've build doesn't transmit any audio. It connects fine without errors in the console and everything works, but no audio. I also don't see any Ice/WebRTC connections in the mumble-web-proxy log. The same if I put your mumble demo server in the address field.

If I use your demo server webrtc client, it connects to my mumble-web-proxy, which then crashes.

It looks like a problem in the js client, but I have no clue why it is not working.

[Johni0702]

Did you npm install after switching branches? That sounds like it might still be using the non-webrtc mumble-client package in the background. To verify, you could grep webrtc dist/index.js, it should look more or less like this:

[streaps]

Yes, I checked several times that it is indeed the webrtc (after I built the master version last night and wondered this morning why the audio is choppy on the raspberry pi). The websocket version does transmit audio.

My index.js looks a bit different, but the webrtc stuff is there.

[Johni0702]

Could you attach a zip of your dist folder or otherwise make it available to me?

[streaps]

it looks exactly the same. grep matched the modules.exports line, because webrtc was in the file path.

[streaps]

mumble-web-dist.zip

[Johni0702]

Hm, yes, they're indeed completely identical except for the line containing the full file path, which you already mentioned.

Were these your attempts to connect from your client to the demo server?

With above client, those should be impossible as it unconditionally sends the webrtc field as part of the Authenticate message but it's missing in above attempts. Only other thing I could think of would be the browser's cache if you've previously had the non-webrtc version at the same URL.

[streaps]

these usernames look like they are from my tests. I'm using incognito sessions, no (caching) proxy and I'm restarting the browser all the time (just to be sure).

I will check the javascript in the browser. thanks for the help! :)

[Johni0702]

Oh, actually, it looks like the mumble-client lib is bundled twice, so those aren't impossible at all. And the fact that opus is disabled would suggest that you're definitely using the webrtc version.

It's bundled twice because of a dependency which mumble-client-websocket has on mumble-client. And it still depends on the old version. Not sure why it would consistently (or even at all) use one bundled version over the other though. I'll have to do some reading up on npm to figure out how to upgrade transitive dependencies.

[Johni0702]

Ok, with above commit (and https://github.com/Johni0702/mumble-client-websocket/commit/d2c0b8069107351034662621ff390b00e3810821), that should now be fixed. Could you try again?

Edit: npm will print a warning about a missing peer dependency during npm install. That's fine, it's not missing, I just don't know how to make npm shut up about it.

[streaps]

it works!

[neumantm]

I have a similiar problem. I compiled and deployed https://github.com/Johni0702/mumble-web-proxy/commit/2a95f11d304d20dca0741ebc9fbc5502fbbf4acb as well as 95dbf255e103e100ecd9e8abf56c68d6315fa918 on my server. Can succesfully connect and see channels and people. But while connected through mumble web I can hear nobody and nobody can hear me.

I also tested connecting with my mumble-web to your test mumble-web-proxy while a friend connects to it using your test mumble-web instance. In this case I could hear him, but he could not hear me.

We are both using firefox.

I also noticed the following in the developper console of firefox:

-- loading localization data for language "en" ... loc.js:144
-- loading localization data for language "en-US" ... loc.js:144
Connecting to server <my-server>/proxy/ index.js:1204
Unhandled data packet: 
Object { name: "CryptSetup", payload: {…} }
client.js:477
Unhandled data packet: 
Object { name: "CodecVersion", payload: {…} }
client.js:477
Unhandled data packet: 
Object { name: "PermissionQuery", payload: {…} }
client.js:477
Connected! index.js:1204
Unhandled data packet: 
Object { name: "ServerConfig", payload: {…} }
client.js:477
Unhandled data packet: 
Object { name: "PermissionQuery", payload: {…} }
client.js:477
ICE failed, add a STUN server and see about:webrtc for more details

Any suggestions?

[streaps]

Could the UDP connection be blocked by any local firewall?

I wonder if the webrtc branch could switch to TCP audio (over websocket), if the webrtc connection fails.

[neumantm]

Thanks for the pointer. I will check that out.

I'm not an expert on webrtc at all but if my "source" is correct, webrtc can theoretically run over tcp only (dunno if this needs to be enabled by the webapp or what).

[neumantm]

Could the UDP connection be blocked by any local firewall?

I'm confused...

Did you mean a UDP connection between the end user and the nginx server, which proxy_passes to mumble-web-proxy(would this really make sense?) or a UDP connection between the ngnix server and the mumble-web-proxy (but the mumble-web=proxy does not even seem to listen on any udp port) or a UDP connection between the mumble-web-proxy and the mumble-server?

[streaps]

I mean the UDP connection between your browser and mumble-web-proxy.

[Johni0702]

WebRTC is built on top of ICE (completely unrelated to murmur's rpc) which can use both, UDP and TCP, at a semi-random (see https://github.com/Johni0702/mumble-web-proxy#firewalls-or-nat) port. It only starts listening when a user connects, which is why you're not seeing mumble-web-proxy listen on any udp ports.

When talking about mumble-web (webrtc branch) behind nginx, there are four connections:

1. browsers to nginx (websocket over tls over tcp)
2. nginx to mumble-web-proxy (websocket over tls over tcp)
3. mumble-web-proxy to mumble-server (mumble over tls over tcp)
4. browser to mumble-web-proxy (ICE over tcp or udp)

The first three are sufficient for connecting to a server as the last one is only used for voice. The error ICE failed, add a STUN server and see about:webrtc for more details also suggests that the last one failed.

What confuses me is

In this case I could hear him

cause I would expect either no or full audio. And the error suggests no audio, so you might be running into two distinct issues.

[neumantm]

Ah I see. Thanks for the clarification. Now I also understand https://github.com/johni0702/mumble-web-proxy#firewalls-or-nat ... I thought the mumble RPC "ICE" was meant here. (Which when thinking about it does not make a lot of sense...) Maybe this could be clarified there.

[neumantm]

What confuses me is

In this case I could hear him

cause I would expect either no or full audio. And the error suggests no audio, so you might be running into two distinct issues.

As I understand it now the original problem should be entirely at the mumble-web-proxy side rather than the mumble-web side, which would suggest this experiment should have worked without any problem. I just repeated this experiment and we could both hear each other. I guess one of us just messed up our audio mixing or something last time.

[neumantm]

After whitelisting the ports in the firewall it works. Thanks!

[neumantm]

After testing for a while, some users reported, that they can't hear other users connected via mumble-web and can't be heard by them either. This seems to only happen sometimes. They can always hear and be heard by users using the client. I'm using ports 50000 to 51000 for ICE and the firewall accepts all UDP packtes from anywhere on the same range.

[streaps]

I guess this is not unusual for this kind of setup. The https/websocket connection to port 443 works most of the time, but a connection to some random UDP port gets blocked by some firewall occasionally. I don't know what happens when the webrtc negotiation fails in mumble-web ...