Closed friederici closed 5 years ago
j1elo
commented
5 years ago Works for me with both KMS 6.7.1 and 6.8.1 & Firefox 62. Are you sure this wasn't a bug in Firefox that got fixed at some point between Firefox 49 and more recent versions?
(by the way Firefox 49 is 2016, a good amount of security bugs have been fixed since then, so using such old versions of this (or any other) browser is probably a huge security risk)
friederici
commented
5 years ago All Versions of Firefox between 49 and current where affected. Just 48 and below works like a charm.
The Setup is: (Ubuntu 18.04 Host with Virtual Box) - vbox NAT - (Ubuntu 16.04 Guest) Port 8888/tcp is forwarded from guest to host
KMS is running inside the guest, the Tutorials and Browsers are running on the host.
When using the old Firefox Versions, an IceCandidate Pair (on the Host IP) will get found and it works. When using current Firefox Versions no working IceCandidate Pair gets found.
(Chrome, Chromium, Opera are ok too)
Could you possible check again with that specific NAT situation? Many thanks!
kylefoley
commented
5 years ago I have this issue as well sometimes with the newer versions FF. Specifically when connecting SFU from FF <-> Chrome
micaelgallego
commented
5 years ago The setup we test extensively is installing KMS in a AWS EC2 instance. Because this setup is the typical production one.
It is possible that installing KMS in a NAT network causes some troubles. Do you have configured STUN? TURN? Where this service is installed? Do you use some public STUN service?
Regards
On Thu, Oct 25, 2018 at 6:11 PM Kyle Foley notifications@github.com<mailto:notifications@github.com> wrote:
I have this issue as well sometimes with the newer versions FF. Specifically when connecting SFU from FF <-> Chrome
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/Kurento/bugtracker/issues/302#issuecomment-433112636, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABBdKBDbHKspo81tVs5oiNXMKkhyka2bks5uoeLLgaJpZM4XznbA.
friederici
commented
5 years ago The setup we test extensively is installing KMS in a AWS EC2 instance. Because this setup is the typical production one. It is possible that installing KMS in a NAT network causes some troubles.
It is required to have KMS behind a NAT for my specific scenario. Since it works with Chrome and old Firefox just fine, I belive it is just a small deviation that breaks the ICE negotiation between KMS and recent Firefox.
Do you have configured STUN? TURN?
I tried both:
the default configuration of KMS (i.e. without adding a STUN server to /etc/kurento/modules/kurento/WebRtcEndpoint.conf.ini
and with adding stunServerAddress=74.125.143.127 stunServerPort=19302 to the above mentioned configuration file. Both cases show similar results. Firefox48 and Chrome are working, Firefox 49-63 are not working.
Where this service is installed?
KMS is running inside a VirtualBox guest (Ubuntu16.04), the Tutorials and Browsers are running on the VirtualBox host system.
Do you use some public STUN service?
I tried with 74.125.143.127:19302, there is no diffence if it is configured or not configured.
From looking at the network traffic between KMS and the browsers, I can tell that chrome gets a Binding Success Response wit a valud XOR-MAPPED-ADDRESS, but the recent Firefox Versions never get such from KMS.
neilyoung
commented
5 years ago In my scenario (KMS in VirtualBox with Ubuntu 16.04, coturn installed aside, Node JS application server and client side ReactJS) there is no problem with FF 63. Is your client also initialized with ICE servers to be used in peer connection establishement?
friederici
commented
5 years ago In my scenario (KMS in VirtualBox with Ubuntu 16.04, coturn installed aside, Node JS application server and client side ReactJS) there is no problem with FF 63.
I dont use a TURN server to connect the client to the KMS. Hence Chrome and old Firefox are happy to negotiate a STUN session, it did not appear to be a requirement to use TURN?
Is your client also initialized with ICE servers to be used in peer connection establishement?
My client is the official Kurento Tutorial, so from what I can tell it appears that chromes uses
(from chrome://webrtc-internals/) iceServers: [stun:stun.schlund.de, stun:stun.l.google.com:19302]
and firefox uses (from about:webrtc) stun4.l.google.com:19302
j1elo
commented
5 years ago TURN is a superset of STUN. Coturn by virtue of being a TURN server is also a STUN server.
If you only configure the server's IP address and port in Kurento's /etc/kurento/modules/kurento/WebRtcEndpoint.conf.ini using the fields stunServerAddress, stunServerPort, then only the STUN part will be used.
If you leave the stunServer... fields commented out and configure access through the turnURL field, then the STUN methods will still be used, and TURN functionality might be used.
Likewise for the browser configuration.
I think the next steps in your case is to analyze the ICE candidate pairs that get selected by each peer, and probably find out that one or both of them is not actually being able to contact the other.
friederici
commented
5 years ago I am not sure how to properly adress the TURN/STUN, local coturn server answers.
Using KMS default configuration and kurento-player default configuration:
Upper (Firefox Version 48) is able to negotiate an ICE pair. Lower (Firefox Version 63) is not able to negotiate an ICE pair.
I don't belive my configuration is wrong, when it plays in Chrome and old Firefox and just fails in new Firefox.
neilyoung
commented
5 years ago Try this https://dragonfly-demo.accuware.com/
neilyoung
commented
5 years ago ... your local ICE server candidates are part of the configuration structure you are passing to the Kurento stack. In JS it could look like so:
let iceServers = await this.iceServerConfig.getIceServerConfig()
let configuration = { iceTransportPolicy: 'all' }
if (iceServers) {
configuration.iceServers = iceServers
}
let options = {
remoteVideo: this.outputVideo,
onicecandidate: this.onIceCandidate,
configuration: configuration,
mediaConstraints: {
audio: false,
video: {
deviceId: this.state.devices[this.state.selectedDevice].deviceId,
aspectRatio: wh[0] / wh[1],
width: {
min: wh[0],
max: wh[0],
ideal: wh[0]
},
height: {
min: wh[1],
max: wh[1],
ideal: wh[1]
},
frameRate: defaultFrameRate
}
}
}
and then...
this.webRtcPeer = WebRtcPeer.WebRtcPeerSendrecv(options, (error, stream) => {
....
This is what I'm doing in my client JS script. I'm asking a server for temporary TURN/STUN configuration. You could of course also hardcode it. But at least you should forward something in the options structure. Not sure, how it looks in Java.
friederici
commented
5 years ago I am talking about an error that is reproduceable with KMS and Tutorials without writing a single line of code.
j1elo
commented
5 years ago Possibly related? (just a quick search)
Could you please test running Firefox in safe mode?
neilyoung
commented
5 years ago Well, ok. You might have overseen, that even the tutorials contain a configuration structure, which have an element "ice_servers", which is left undefined, since it is not necessary in 80% of the cases.
var args = getopts(location.search,
{
default:
{
ws_uri: 'wss://' + location.hostname + ':8433/kurento',
ice_servers: undefined
}
});I don't say, that it will fix your issue, but as you can see, there is a running solution based on KMS in Firefox 63.
But why bother...
friederici
commented
5 years ago Could you please test running Firefox in safe mode?
I tried in safe mode, the result is the same. Firefox v48 plays, Firefox v49 and v63 do not:
v48 and v49 side by side (in safe mode)
v63 (in safe mode)
friederici
commented
5 years ago Well, ok. You might have overseen, that even the tutorials contain a configuration structure, which have an element "ice_servers", which is left undefined, since it is not necessary in 80% of the cases.
I tried to search for this configuration structure but i cannot find anything similar in the Kurento Java Tutorials:
$ grep -rnwi 'kurento-tutorial-java/' -e 'ice_servers' $ grep -rnwi 'kurento-tutorial-java/' -e 'iceservers'
Are you referring to something that is specific to the NodeJS tutorials?
neilyoung
commented
5 years ago I was only talking about JS
j1elo
commented
5 years ago I can confirm the issue under the conditions described.
10.0.2.15).This happens to work with Chrome / old Firefox because KMS is able to find its own "public" IP address (from outside the NAT) when it discovers its own Peer-Reflexive ICE Candidate, during the ICE Connectivity Check stage with the browser.
When KMS receives the remote candidate from the browser (with IP 172.29.21.177), it performs the connectivity check (a ping/pong style message + reply, actually a STUN Bind Request + Response). The check is successful because it's possible to reach that IP from within the NAT. It also tells the browser about where KMS can be accessed, so the browser directs there its response. This response includes both the IP addresses of the public and private sides of the NAT. When KMS receives the connectivity response, it finds out what is its own public-side IP, so now it knows how to communicate with the external peer (the browser).
Having a clear understanding of ICE connectivity checks and peer-reflexive candidates is required to understand all this.
Thanks to this peer-reflexive candidate, the peer inside a NAT (in this case, KMS) is able to know what is its own IP address from outside the NAT. Which is necessary for ICE to success and the stream to flow. Problem is that starting from Firefox 49, that peer-reflexive candidate is nowhere to be found.
Verify by running KMS then following the logs:
tail -f /var/log/kurento-media-server/*.log | grep 'prflx'While running Chrome:
/usr/bin/chromium-browser \
--user-data-dir="$(mktemp --directory)" \
--enable-logging=stderr \
--vmodule='*/webrtc/*=2,*/libjingle/*=2' \
--v=0 \
"https://localhost:8443/" \
2>&1 | grep 'prflx'And lastly while running Firefox:
export R_LOG_DESTINATION=stderr
export R_LOG_LEVEL=7
export R_LOG_VERBOSE=1
/usr/bin/firefox \
-profile "$(mktemp --directory)" \
-no-remote \
"https://localhost:8443/" \
2>&1 | grep 'prflx'In my test, there is no peer-reflexive candidate found with Firefox, but there is one when trying with Chrome. It can be seen both inside the NAT (in KMS logs) and outside the NAT (in Chrome logs).
Now we are wondering why there is no peer-reflexive candidate found starting with Firefox 49. This is possibly a Firefox bug, maybe?
Next step is you could use Wireshark to inspect the connectivity check stage, and have a clear picture of whether there is an actual outbound check going from KMS to the browser, that should trigger creation of a peer-reflexive candidate in the browser. I'd assume this is actually happening, or else it wouldn't work with Chrome.
So following this examination, a further step would be to see why Firefox doesn't create a peer-reflexive candidate when the connectivity check response arrives from KMS. Possibly asking in Mozilla support forums and/or opening a bug report with sufficient investigation results.
From our point of view this is totally opposite to what is the standard installation practice for Kurento Media Server, so we won't follow through as it falls totally out of scope for this project. If this scenario is an important part of your workflow and you cannot or don't want to spend the time researching the problem, please contact us for more specific commercial support.
neilyoung
commented
5 years ago To get this right: Scenarios, in which there is a strict separation between AS and client (e.g. something like this: KMS and AS are running in the cloud, while the client is just talking to the AS) are not hit? This would match my observation, since I don't have any problem with that.
j1elo
commented
5 years ago @neilyoung Correct, the typical use case which forms the 99% is browser clients (Chrome, Firefox) needing to open their own NAT ports, while KMS is freely accessible. Not the other way around.
Note that the client may communicate with the Application Server for signaling purposes, but at the end of the day the important communication is done between the WebRTC engines of the browser and KMS.
Also note that in reality you can have all NAT firewalls you want, even deploying KMS behind a NAT firewall... but as any WebRTC implementation would require: each NAT'ed peer MUST have access to some STUN or TURN servers. You can indeed have KMS behind a NAT, as long as you configure some STUN or TURN server that lives outside and aids during the ICE connectivity process.
That's the correct way of doing WebRTC. Relying on peer-reflexive candidates that happen to work, that kind of stuff, might work for some specific and very limited situations, but at the end of the day the general rule is clear: every peer behind a NAT needs an outside-of-the-NAT STUN/TURN server to help open its NAT ports.
neilyoung
commented
5 years ago you can indeed have KMS behind a NAT, as long as you configure some STUN or TURN server that lives outside and aids during the ICE connectivity process.
In my scenario I'm having TURN and STUN on the same machine as KMS. Had no problems with that up to now. The client obtains one time credentials for the TURN server and STUN configuration (addresses, port) by a REST call to the AS (which generates that). So basically both the KMS and the client use the same TURN/STUN configuration. AS and KMS are hosted at Amazon EC2.
neilyoung
commented
5 years ago To be more specific: The KMS and the client use the public IP of the COTURN instance for sure
j1elo
commented
5 years ago That's actually the preferred deployment model in OpenVidu.
It works because KMS is sitting in a publicly accessible machine, no NAT to be traversed by incoming connections. So Client --> KMS streams don't need to traverse any NAT. But KMS --> Client streams do need to traverse the client's NAT, that's why the client still needs to use some STUN server. Having the STUN server together in the same machine as KMS just makes things simpler and cheaper.
neilyoung
commented
5 years ago Thanks for this detailed insight.
friederici
commented
5 years ago Oh I see. Thank you for the detailed explanation! So it was just by coincidence that KMS could pick up the external (inner-NAT-sandwich) address.
I've checked that it works, when configuring an STUN server that is located on the VM Host (along with the application server and browser in my case). I was not aware that configuring a public STUN server would not reveal the required address in my specific setup.
j1elo
commented
5 years ago Motivated from the conversation in this issue, I've added a new section to the FAQ, also linked in the installation guide: When is STUN needed?
KMS Version: Both below show the same behavior:
AND
Other libraries versions: Both below show the same behavior:
AND
Client libraries
Browsers tested
System description: KMS is located in an VirtualBox VM Guest Java Tutorials and Browsers are running on VirtualBox Host
What steps will reproduce the problem?
What is the expected result? Video should start playback.
What happens instead? Console: ICE failed, see about:webrtc for more details
Does it happen with one of the tutorials? You can reproduce it with kurento-player but I am pretty sure all of them are affected.
Please provide any additional information below.