Hubs-Foundation / hubs

Duck-themed multi-user virtual spaces in WebVR. Built with A-Frame.
https://hubsfoundation.org
Mozilla Public License 2.0
2.13k stars 1.41k forks source link

TURN server doesn't work with Firefox > 86 #4187

Closed vincentfretin closed 3 years ago

vincentfretin commented 3 years ago

Description On some networks where you need the TURN server, you can enter the room, you can see the other avatars, the volume mic is working, but you can't hear the others participants (their avatar head is not moving), and they can't hear you when using FF 87 or 88 on Ubuntu 18.04. There is no errors, and you are not disconnected because of the bad WebRTC connection where no ice candidate was selected.

To Reproduce Steps to reproduce the behavior:

  1. Go to a room with "?force_turn=true" in the url so that it forces iceTransportPolicy: relay to go through the coturn server
  2. Enter the room
  3. look at ice candidates with about:webrtc

There is no candidates on Firefox 87 and 88.

Screenshots Tested on my machine with Ubuntu 18.04 Firefox 86 where it was still working Capture d’écran de 2021-04-24 14-27-31

Firefox 87 and Firefox 88: Capture d’écran de 2021-04-24 16-00-38

Hardware

Additional context It seems you allow only relay-tls candidates on your coturn server because you have no-udp and no-tcp https://github.com/mozilla/hubs-ops/blob/4a353b156f123262026af3450e8f91b60f9709f5/ansible/roles/janus/vars/local.yml#L12-L13

If you don't add those no-udp and no-tcp options in your turnserver.conf, you will have relay-udp and relay-tcp candidates and this will probably fixes the issue. I had the same in my coturn conf with my networked-aframe project and I commented them and this fix the issue. This is when I tested with xirsys TURN server that I found I get relay-udp and relay-tcp candidates with their TURN server, so I wondered why I didn't have those with coturn, and this was because I got the no-udp and no-tcp in the config.

On my coturn server (with no-udp and no-tcp), I did a tcpdump/wireshark test with FF 86 and 87. On FF 87, it opens the TCP socket for the relay-tls candidate but stop there, no further packet, it doesn't do the TLS Client Hello at all, and the candidate is not showing in the table in about:webrtc

vincentfretin commented 3 years ago

Currently I think you redirect 80 to 5439 via iptables if I remember what I saw in hubs-ops. 5439 or 3478 is the same for coturn, it listens for all UDP/TCP/TLS on the two ports.

You may need two additional urls for the iceServers:

turn:agitated-elf.reticulum.io:80?transport=udp
turn:agitated-elf.reticulum.io:80?transport=tcp

for the relay-udp and relay-tcp to work. Currently you have turns:agitated-elf.reticulum.io:80 and turns:agitated-elf.reticulum.io:80?transport=tcp. For me it's duplicate, the first will also select a relay-tcp or relay-tls (you maybe thought it would be udp?). You should keep only the second for me, and use 443 instead of 80 because some network find weird to have TLS on 80 and block it (there is an issue somewhere about it)

From my testing on my coturn server, I tested each url one by one and for each browser in latest version I get the following:

"turn:turn.example.com:3478?transport=udp", # FF ok (relay-udp), Chrome ok (relay-udp), Safari nok
"turn:turn.example.com:3478?transport=tcp",  # FF ok (relay-tcp), Chrome ok (relay-tcp), Safari ok
"turns:turn.example.com:5349?transport=tcp",  # FF ok (relay-tls on FF 86, relay-tcp on FF 87+), Chrome ok (relay-tls), Safari nok
"turns:turn.example.com:443?transport=tcp", # through traefik tcp passthrough to 5349, works only on Chrome (relay-tls) and Firefox 86 (relay-tls)

For reference xirsys gives all those urls:

"stun:eu-turn1.xirsys.com",
"turn:eu-turn1.xirsys.com:80?transport=udp",
"turn:eu-turn1.xirsys.com:3478?transport=udp",
"turn:eu-turn1.xirsys.com:80?transport=tcp",
"turn:eu-turn1.xirsys.com:3478?transport=tcp",
"turns:eu-turn1.xirsys.com:443?transport=tcp",
"turns:eu-turn1.xirsys.com:5349?transport=tcp"
keianhzo commented 3 years ago

Thanks @vincentfretin for the thorough report as always. I'm not sure if we want to add support for non secured TURN. This bug it's very specific to FF<89, it's already fixed in Nightly and will be available soon in upstream stable so I think it's worth waiting at this point.

keianhzo commented 3 years ago

I'm closing this as it's already fixed in stable FF (v89)