search

webrtc / samples

WebRTC Web demos and samples
https://webrtc.github.io/samples
BSD 3-Clause "New" or "Revised" License
13.85k stars 5.7k forks source link

RTCDataChannel examples don't work on iOS 11/12 #1123

Closed thehunmonkgroup closed 2 years ago

thehunmonkgroup commented 5 years ago

Browser affected

Safari on iOS 11 and 12

Description

All of the examples under the 'RTCDataChannel' section of samples fail to work in Safari on iOS 11/12. They work in Safari on MacOS, and in other browsers.

Steps to reproduce

  1. Visit any of the following examples in Safari on iOS 11/12 and run them:

Expected results

Data is sent between the peer connections.

Actual results

No data is sent between the peer connections.

thehunmonkgroup commented 5 years ago

Filed https://bugs.webkit.org/show_bug.cgi?id=189503

kelvinkoko commented 5 years ago

Those samples are not working in my Mac's Safari Version 11.1.2 (13605.3.8). What is the minimum version for those to work on safari?

thehunmonkgroup commented 5 years ago

The first three examples work on 12.0 (13606.2.11), as I recall all examples worked fine on the previous version of Safari 11 I had installed.

lgrahl commented 5 years ago

Please create a separate issue if one specific data channel example doesn't work when other data channel examples do work.

kelvinkoko commented 5 years ago

@thehunmonkgroup Thanks for the reply, I will further check whether it is related to other setting or it is not working on a specific version @lgrahl OK, I will create separate issue to provide more details after more checking

thehunmonkgroup commented 5 years ago

OK, so I’ve confirmed data channels DO work in Safari on iOS, but there’s a caveat: iOS does not include local ICE candidates by default, and many of the data channel examples I’ve seen (including the examples in this repository) depend on that, as they’re merely sending data between two peer connections on the same device.

See https://bugs.webkit.org/show_bug.cgi?id=189503#c2 for how to temporarily enable local ICE on iOS.

We could close this issue, but before I do, wondering if there's some kind of intelligent error message that can be provided in this case?

I've opened #1130 to address the other bug reported in this issue.

lgrahl commented 5 years ago

We could close this issue, but before I do, wondering if there's some kind of intelligent error message that can be provided in this case?

We could show an error that the peer connection failed. This would be useful for all examples if not already implemented.

anderspitman commented 5 years ago

@thehunmonkgroup thanks for your work on this. Unfortunately I haven't been able to get your solution to work. When attempting to call navigator.mediaDevices.getUserMedia({audio: true, video: true}) my ipad is saying it's not allowed and possibly disabled by the user. Have you seen this before?

Also, can you explain more why calling getUserMedia enables local ICE candidates in the first place?

lgrahl commented 5 years ago

Also, can you explain more why calling getUserMedia enables local ICE candidates in the first place?

The implementers (read: browser vendors) of WebRTC thought it would be reasonable to assume that leaking all host ICE candidates (read: all private IP addresses, including those of VPNs) is perfectly fine when permission for audio/video capture has been granted but not without that permission. Chrome and Firefox do hand out the default route's host ICE candidate but Apple took that to an extreme and does not hand out any host ICE candidates before this permission has been granted.

IMHO the decision to couple this with getUserMedia was highly questionable and it's even more questionable that one-way media and data use cases still have no dedicated way to request permission that is not bemusing to the user: Take a selfie before you can establish a connection... oh you have no audio or video device? Too bad. :slightly_smiling_face:

thehunmonkgroup commented 5 years ago

my ipad is saying it's not allowed and possibly disabled by the user

Nope, never seen that before. I'd test with other devices to see if it's isolated to that device.

anderspitman commented 5 years ago

@lgrahl maybe I'm missing something here. WebRTC is often touted as a UDP-like solution for HTML games, which is exactly what I'm working on. How can that ever be viable if the user has to grant permission to their webcam in order to send game packets? Is there really no way to just open data streams?

anderspitman commented 5 years ago

Ok, the mDNS/ICE draft sheds some light on this, and at least it's explicitly acknowledged that this creates issues for data channels. But this seems overly complicated. Why not just end-to-end encrypt the sensitive portions of the offer/answer so that neither the local JS or the signaling server can access them? In my case I don't care what the user's IP address is as long as the clients can establish a connection over a LAN. I'm sure it's not that simple but just trying to get a handle on the situation.

anderspitman commented 5 years ago

Sorry for the comment spam. If a STUN server is used, does it always get access to the host candidates? That might be a good workaround for me. Even though my application is LAN-only, I still need a signaling server anyway. If I can get the candidates by running my own STUN server I think that could work.

lgrahl commented 5 years ago

How can that ever be viable if the user has to grant permission to their webcam in order to send game packets?

I welcome you to my world. :slightly_smiling_face: A more appropriate permission request is all I want and more voices to support that are always welcome.

Why not just end-to-end encrypt the sensitive portions of the offer/answer so that neither the local JS or the signaling server can access them?

Because how would the keys be exchanged if there is no connection and you need the ICE candidates to set one up? :slightly_smiling_face: That's what the mDNS thing is working around. But it's not going to work everywhere.

If a STUN server is used, does it always get access to the host candidates?

Only if you can deploy it in your LAN.

anderspitman commented 5 years ago

I welcome you to my world. slightly_smiling_face A more appropriate permission request is all I want and more voices to support that are always welcome.

Happy to provide my input. Where's the best place to join the conversation?

Because how would the keys be exchanged if there is no connection and you need the ICE candidates to set one up?

Through the signaling server? Even as I write that I know it can't be part of WebRTC because signaling isn't specified.

Only if you can deploy it in your LAN.

Really? What's the difference between being accessed on a LAN and being accessed over the internet, from the STUN servers perspective?

lgrahl commented 5 years ago

Happy to provide my input. Where's the best place to join the conversation?

Good question.

Through the signaling server?

Chicken-egg problem since the application handles the signalling, so it has access to the data. :slightly_smiling_face:

What's the difference between being accessed on a LAN and being accessed over the internet, from the STUN servers perspective?

The IP address the STUN server sees is different. While it may work if you have internet access, there's a chance that both peers in the LAN can't connect to each other because your router may not support NAT hairpinning. Or if you're behind an ISP that uses DS-Lite, the data will go to the DS-Lite gateway and back again (if it works at all).

anderspitman commented 5 years ago

Good question.

* The [issue mentioned above](https://github.com/w3c/webrtc-pc/issues/2012) is a good start.

* The [WebRTC mailing list](https://lists.w3.org/Archives/Public/public-webrtc/).

* [w3c/webrtc-nv-use-cases#1](https://github.com/w3c/webrtc-nv-use-cases/issues/1)

Awesome, thanks!

Chicken-egg problem since the application handles the signalling, so it has access to the data. slightly_smiling_face

I was thinking the browser WebRTC implementation would do the encryption, then pass the encrypted data up to the application to pass on to the peer. Unless the app exploits the browser it shouldn't be able to crack the messages. I'm no security expert though, so I defer to you and have no doubt this has been thought through.

The IP address the STUN server sees is different. While it may work if you have internet access, there's a chance that both peers in the LAN can't connect to each other because your router may not support NAT hairpinning. Or if you're behind an ISP that uses DS-Lite, the data will go to the DS-Lite gateway and back again (if it works at all).

Ah ok. I'm not very familiar with STUN, so I wasn't sure if the browser implementation provided the server with extra information that wasn't available to JavaScript. Doesn't sound like it.

szimek commented 5 years ago

I've got exactly the same issue. Long time ago I created a shameless AirDrop clone (https://www.sharedrop.io) and while I managed to fix it just enough to make it work in desktop Safari (it uses a really old version of PeerJS library), it doesn't work on iOS devices, because of the issue mentioned here. Getting users to grant permissions to their mic and camera just be able to send a file, will be really hard to explain. In this particular case it actually results in less security for users... A new permission request (though I have no idea how one can explain it in one sentence to non-tech-savvy people) or a browser setting would be really great.

That said, I've got slightly related question. When using an iOS device or when I'm behind a VPN, the data channel examples, e.g. filetransfer, do not work, but they also don't show any error message, even in JS console. I've got exactly the same issue in my app and I don't know if there's any way to figure out that something went wrong and somehow inform users about it. Any ideas?

lgrahl commented 5 years ago

@szimek Yes, we (I believe) agree with you. But this is not the place for complaining about this problem. I mentioned several more appropriate places or your voice (which is needed) will not be heard. :slightly_smiling_face:

For your issue: Bind the iceconnectionstatechange event and wait for failed.

vitobid commented 5 years ago

I have data channels working on IOS 11/12. I have found that when trying to connect IOS devices you must do the handshake both ends. In all other cases, only 1 browser needs to send the offer and the other sends the answer. Both will find (hopefully) ICE candidates and the oniceconnectionstatechange will fire to let you know that the connection is completed. In IOS no built in callback functions fire so you have to manually check if the channel is open. How i got it to work is to get the host to send the offer first and when the answer has been received then make the other client send an offer. In IOS 11/12 when both clients have sent and received offer/answers then the channel will be open and you can send data. Also you have to check for disconnection manually because as i said no callbacks from the RTC library fire. But it does work with a little time and effort. I can confirm that it only works reliably over a LAN connection and trying to connect IOS via public internet is hit and miss depending on the mobile network provider. I fall back to websocket connection to communicate in the scenario where 2 connections don't open after 3 seconds of the last answer being received. It's the only way i could get it to be production friendly so it works for everyone all the time. If you want, i'm happy to post code here but i am quite busy and i only commented here because i wish someone had written what i have taken a month to learn by trial and error. Fingers crossed for ios 13!

normal scenario (non ios) client 1 sends offer client 2 receives offer and sends answer client 1 receives answer oniceconnectionstatechange will tell you that its connected Now both clients are connected and can transmit data.

IOS 11/12 client 1 sends offer client 2 receives offer and sends answer client 1 receives answer and sends a message back to client 2 to say you go next client 2 sends offer client 1 receives offer and sends answer (no status change event will fire from RTC) Now both clients are connected and can transmit data.

vitobid commented 5 years ago

If anyone wants to see this in action, right now my game is still in development so you would need to message me. When it is live i will post a link here.

anderspitman commented 5 years ago

@vitobid this is a very interesting result. If I'm not mistaken, this would be considered a bug in iOS Safari since their intention is that host ICE candidates should only be available in an mDNS environment. @lgrahl does that match your understanding?

vitobid commented 5 years ago

I am unsure if this is intentional on apples behalf or not because there is still the problem of IP leakage and it's possible that apple are preventing ICE candidates on this basis. Interestingly, even when you go into safari settings and allow ICE candidates they don't work as of ios 12 so i would say that's a bug. I can't offer any more info tho sorry.

anderspitman commented 5 years ago

When you say enable ICE candidates do you mean host candidates specifically or all ICE candidates? I thought iOS works with normal ICE candidates from STUN server interactions. I didn't even realize Safari had a setting for this.

vitobid commented 5 years ago

In IOS 12 you can go to settings->safari->advanced->experimental features->Enable MDNS ICE candidates

vitobid commented 5 years ago

safari wont work with Stun as far as i have seen. It always defaults to Turn in IOS for me.

anderspitman commented 5 years ago

Interesting, I had thought mDNS ICE was enabled by default. I wonder how long it will be before all the browser vendors have mDNS working on stable. Should solve my use case at least. For now I have to use the webcam permission hack.

vitobid commented 5 years ago

I couldn't get video to work on ios at all. The getusermedia() function wasn't working for me at the time i couldn't even get the camera of the phone to display to myself.... i had no really use for it at the time anyhow i was just experimenting. I didn't learn much as i wasn't successful right from the start and so didn't persevere. I use the data channels for a rt multiplayer game i'm developing which is why i have spent so long trial and error'ing the data channels.

anderspitman commented 5 years ago

Yeah I went down that rabbit hole as well. The solution for me was to serve the app over an HTTPS connection, because Safari doesn't allow webcam/mic access except over HTTPS.

My use case is also a game, specifically for LAN play.

JustMaier commented 5 years ago

@anderspitman and @vitobid did you finally find a decent workaround? I'm trying to create a datachannel between iOS Safari and Chrome on Windows and not having much luck. iOS will prepare the offer, but it never seems to do anything with the answer that it receives. If I enabled mDNS ICE it seems to work fine between iOS devices, but still doesn't work between iOS and desktop.

I've also tried doing the getUserMedia() workaround on https with a wss signal server, but it didn't seem to work. Unfortunately, I'm working on Windows so I'm flying kind of blind when it comes to debugging iOS Safari.

anderspitman commented 5 years ago

@JustMaier unfortunately I only made it as far as proving to myself that it could work, since I didn't want to get all the way to the end of writing my game only to have this be the blocker. Even the prototype required a ton of hacking and fiddling, but ultimately getUserMedia() + WSS were the big keys, as I recall. Honestly, the long term solution is mDNS. If you need something working today I'm pretty sure it's going to involve getUserMedia, which is a nasty hack for a data channel.

anderspitman commented 5 years ago

One thing I'd recommend is actually looking at the offers coming through. IIRC they don't always have the local IPs, and there's no way for them to connect if that's not included (without mDNS). So if there's no local IP in the offer you know something is wrong. That's going off memory though so might not be accurate, sorry.

lgrahl commented 5 years ago

I believe using a STUN server would solve most of your issues. There are plenty that are publicly available. It's not the ultimate solution and has drawbacks but at least it would work for many.

Also, mDNS has drawbacks too and it's unlikely to solve your issues without usage of a STUN and TURN server. However, the ultimate solution would be https://github.com/w3c/webrtc-pc/issues/2012 plus usage of a STUN and TURN server.

JustMaier commented 5 years ago

One thing I'd recommend is actually looking at the offers coming through. IIRC they don't always have the local IPs, and there's no way for them to connect if that's not included (without mDNS).

You're correct. After reviewing the offer/answer from the iOS device it's clearly different than the offers from Chrome. Interestingly enough, if Bonjour is installed on the Windows device, it connects fine. I assume that means that it'd connect fine with Mac devices as well since they have mDNS by default.

I believe using a STUN server would solve most of your issues. There are plenty that are publicly available. It's not the ultimate solution and has drawbacks but at least it would work for many.

I'm using STUN and TURN servers but I seem to be having the same problems still, maybe I'm doing it wrong?

new window.RTCPeerConnection({
    iceServers: [
        { urls: 'stun:stun.l.google.com:19302' },
        { urls: 'stun:global.stun.twilio.com:3478?transport=udp' },
        {
            url: 'turn:turn.anyfirewall.com:443?transport=tcp',
            credential: 'webrtc',
            username: 'webrtc'
        }
    ]
})

After adding the TURN server, the iOS device doesn't seem to signal an Answer or Offer.

lgrahl commented 5 years ago

Perhaps you're affected by the dual-stack bug? Otherwise, it must be another issue which should be reported. In any case, please raise your voice towards them.

JustMaier commented 5 years ago

Maybe. I'll try to inspect the Safari console and get back to you.

JustMaier commented 5 years ago

Well, finally got it to work with wss for signaling and GetUserMedia. So, guess that workaround works as expected, it's just a bummer that to even do a simple data connection you have to request media permissions or use mDNS which isn't supported by any other types of devices.

Seems like a common alternative is a relay, kind of a bummer :(

lgrahl commented 5 years ago

If you really don't discover any srflx candidates, that's a bug.

mrmiguu commented 5 years ago

Trying to connect macOS Mojave Safari Version 12.1 (14607.1.40.1.4) to iOS Safari/604.1.

I can get the RTCDataChannel communicating with the getUserMedia({audio:true}) trick, but not without it.

I'm within my LAN and am using RTCConfiguration:

{
  iceServers: [
    {urls: 'stun:stun.services.mozilla.com'},
    {urls: 'stun:stun.l.google.com:19302'},
  ]
}
lgrahl commented 5 years ago

You folks really should report that on bugs.webkit.org and not in this issue. There's nothing that the samples can do about this.

vitobid commented 4 years ago

I have a fully working script which will work on all platforms / OS, client + server script. https supported. Contact me at Tegridy Games if you want a copy.

anderspitman commented 4 years ago

Any reason you can't drop it in a gist or github repo?

On Sun, Sep 15, 2019 at 3:48 PM David Colemn notifications@github.com wrote:

I have a fully working script which will work on all platforms / OS, client + server script. https supported. Contact me at Tegridy Games if you want a copy.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/webrtc/samples/issues/1123?email_source=notifications&email_token=AB3VHKD5WMNPBYCQRFIMWGTQJ2UTXA5CNFSM4FUOTS52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6XZW7A#issuecomment-531602300, or mute the thread https://github.com/notifications/unsubscribe-auth/AB3VHKHEI47T2I3DFX42T33QJ2UTXANCNFSM4FUOTS5Q .

juberti commented 4 years ago

@vitobid can you share more details?

fippo commented 2 years ago

at least the basic sample works so there is nothing actionable here anymore

JayeshMardiya commented 1 year ago

let candidateJson = ["command": "message", "content" : textValue, ] as [String : String] self.client.sendData(data: candidateJson.json.data(using: .utf8) ?? Data.init(capacity: 1), binary: false)

@fippo While I am sending data as a dictionary, It's not working