Connection drops immediately after connecting

[domschiener]

Currently using PeerJS with Meteor (here is the project: https://github.com/domschiener/peered). Everything works fine locally on the same connection, but as I deploy it to my server and try to connect with a friend, the connection is very unstable and oftentimes drops immediately after connecting.

Anyone have an idea as to why that could be?

PeerJS:  Socket open
Success! Your Peer ID is: Ajrgmd6A3KwDLnuoB
PeerJS:  Creating RTCPeerConnection.
PeerJS:  Listening for ICE candidates.
PeerJS:  Listening for `negotiationneeded`
PeerJS:  Listening for data channel
PeerJS:  Listening for remote stream
PeerJS:  Setting remote description RTCSessionDescription {type: "offer", sdp: "v=0
↵o=- 6918372162384193671 2 IN IP4 127.0.0.1
↵s…id:data
↵a=sctpmap:5000 webrtc-datachannel 1024
↵"}
Received new connection ERQFHRAHkFDRMLnJf
PeerJS:  Set remoteDescription: OFFER for: ERQFHRAHkFDRMLnJf
PeerJS:  Created answer.
PeerJS:  Set localDescription: answer for: ERQFHRAHkFDRMLnJf
PeerJS:  Received ICE candidates for: ERQFHRAHkFDRMLnJf
PeerJS:  Added ICE candidate for: ERQFHRAHkFDRMLnJf
PeerJS:  Received data channel
PeerJS:  Data channel connection success
PeerJS:  DataChannel closed for: ERQFHRAHkFDRMLnJf
PeerJS:  Cleaning up PeerConnection to ERQFHRAHkFDRMLnJf
Connection closed```

[Bobisback]

This is what it outputs when it is not working in safari. Aka this browser is receiving the call.

"Added ICE candidate for:" – "3b5aflyiazv00000" – "candidate:1737421303 1 udp 1686052607 216.10.85.7 52681 typ srflx raddr 192.168.102.146 rport 52681 generation 0 ufrag ZuhX network-id  1 network-cost 10" (vendor.js, line 88282)

Here is the same safari browser when it is working, aka it is making the call.

"Added ICE candidate for:" – "3b5aflyiazv00000" – "candidate:239386166 1 udp 2122260223 192.168.102.146 62277 typ host generation 0 ufrag cr/a network-id 1 network-cost 10"

What is really interesting is the fact that When Safari is calling the other browser this line of code never runs, and it still receives the ice candidate correctly. Maybe the problem is that the peer making the connection to Safari is not adding a ice server when it should be and that is what is killing safari, it is waiting for something that is never added?

When I connect a chrome browser to a firefox one here is the output on firefox.

PeerJS:  Added ICE candidate for: 3b5aflyiazv00000 candidate:1737421303 1 udp 1686052607 216.10.85.7 50742 typ srflx raddr 192.168.102.146 rport 50742 generation 0 ufrag Vl9c network-id 1 network-cost 10
PeerJS:  Added ICE candidate for: 3b5aflyiazv00000 candidate:1086800582 1 tcp 1518280447 192.168.102.146 9 typ host tcptype active generation 0 ufrag Vl9c network-id 1 network-cost 10

And the output on chrome.

PeerJS:  Added ICE candidate for: 77wi7q4ssif00000 candidate:0 1 UDP 2122252543 192.168.102.146 57094 typ host
util.js:65 PeerJS:  Added ICE candidate for: 77wi7q4ssif00000 candidate:2 1 TCP 2105524479 192.168.102.146 9 typ host tcptype active
util.js:65 PeerJS:  Data channel connection success
logging.service.ts:11 Data connection open according to peerjs.
util.js:65 PeerJS:  Added ICE candidate for: 77wi7q4ssif00000 candidate:1 1 UDP 1686052863 216.10.85.7 57094 typ srflx raddr 192.168.102.146 rport 57094

So for some reason on safari is just not passing the ice candidate correctly. On fire fox I just got a new error that said ice failed add a turn server. So I still need one even though it is local ip's?

How hard are they to setup? I have a nodejs server atm, can I set one up on that?

Any thoughts?

Edit: Interesting, if I call firefox from safari I consistently get a negotiation of connection failed, and firefox gets the add a turn server message. I guess the next step is to setup a turn server?? I was trying to avoid that :(

[abardik]

@Bobisback, You don't have to setup your own TURN for testing purposes (look at the comments above about numb.viagenie.ca).

Maybe it can be usefull for somebody, so I'll try to simplify some webrtc details here.

The webrtc idea is to connect two devices directly to each other the shortest and fastest way. If devices on the same host, the first candidate (127.0.0.1 typ host) is used. This candidate is available anytime and anywhere if your device's network adapter is not disabled. And any clients on this one host can connect to each other directly. If your devices in the same network, another candidate will come in help (local_network_address typ host). Webrtc clients use this type of candidates next after localhost. And it works because all of your local network devices have a 'physical' connections between each other. In other words, if you can ping other computer, you can reach it with webrtc.

Next, if your devices in different networks or network segments, including internet, they can't ping each other (and connect to each other with webrtc or anything else) if they don't have a direct public IP address (dynamic or static, it doesn't matter). The examples are everywhere: any office computer, most of home computers behind Wi-Fi router, etc. In this situation, STUN with UDP hole punching technique is coming to help. To be pretty simple, your device makes a request to STUN and receives a response with global IP:port, with which it connected to STUN (this IP:port is your internet provider's NAT IP:port which is used to transfer your requests and receive responses from web servers). Today, by statistics, in ~80% situations regular NATs will receive packets and transfer it to your device not only from hosts you requested, but from any other hosts. So, you can send this IP:port to another webrtc client, and that client can connect to this IP:port and send you any data. Once again, this is an external well-reached from anywhere IP:port of your provider's NAT (not yours), which is used then to transfer incoming packets to your host only. This is called a UDP hole punching (you can google a detailed explanation with examples). And this IP:port from STUN is a 'srvflx' candidate in your webrtc logs.

But in some situations (~20%) NAT accepts packets on that IP:port only from the same host you sent request to, or uses new port for every single request-response. So, if you send a request to google STUN (and NAT assign some port to you for this request), it will accept packets (or even just one response) on this port from google STUN only and from nobody else. This kind of NATs are called 'symmetrical NATs'. In this case, your webrtc client will try to use 'srvflx' candidate and will fail. And here we have TURN.

TURN provides public IP:port for you to send it to other webrtc clients as your 'relay' candidate. Webrtc clients will try to use 'relay' candidates only if they failed to connect with 'host' and 'srvflx' candidates. Actually, the algorithm a little bit smarter and uses candidates based on not only 'typ', but on 'network-cost' and other parameters. TURN works like your company's NAT and provides a real always reachable public IP:port (dynamic or static), but without 'symmetrical issue'. But all of your trafic is going through your TURN now. So, you have to place your TURN in well accessable locations, such as AWS infrastructure.

The last thing about establishing webrtc connection is a 'trickle ICE'. This technique is invented to reduce a connection time. Without it webrtc client sends SDP only when it got all candidates from its network, STUN and TURN servers. This collection work could take a lot of time (seconds). So, with trickle ICE enabled, webrtc client sends SDP as soon as it obtained its first candidate (which is localhost typ host). Another peer can reach this candidate if they both works on the same host. So, it makes sense to send an SDP immediately. Then, local network candidates are obtained (it's very fast) and sent, and it's possible to connect to all computers in your local network (no delay, practically). If your peers in diffefrent networks, the 'srvflx' candidate should be obtained from STUN (1-2 seconds) and used to try to connect. And, finally, TURN candidates (another 1-2 seconds). So, to avoid this delay, webrtc clients try to connect as soon as possible and then they send additional candidates, and another peers try to connect with those new candidates until they are connected. Even if a connection is already established, webrtc clients can use another candidates if they decided (based on network-cost parameter, for example) that new candidate is better then old one. Sometimes this technique fails, and we have dropped connections in a few seconds after successful connect.

So, if all of your devices are in the same network segment (wi-fi or ethernet), you don't have to use any STUN or TURN, because all of your devices are 'pingable' to each other. But, if you have some kind of personal or corporate firewall or something that can block incoming or even outgoing trafic on your devices, you could hit this (fire)wall. I never experienced this kind of problems before, but theoretically it could be.

Try to find any pure webrtc example (no peerjs or other libraries). If you have no netwrok limitations you will connect in any direction. And then you can continue your experiments with peerjs.

[abardik]

I just tried all of my peerjs clients (windows chrome, windows electron, mac electron, windows nodejs, android chrome, android phonegap, ios safari, ios phonegap) with and without TURN/STUN. All of them work in any direction when in local Wi-Fi network with zero ICE configuration (iceServers: []). Then, I tried LTE on iOS, and it works fine without TURN and even without STUN. So, my carrier provides public IP addresses for LTE clients, cool. Then, I tried Wi-Fi hot spot on iOS device and connected Android client to LTE through iOS wi-fi hot spot. In this configuration Android didn't work without TURN. So, Wi-Fi hot spot acts like a symmetrical NAT or proxy. But in Wi-Fi network all clients just work without any TURN or even STUN, because all of them can reach each other directly.

[Bobisback]

Thanks for all the great information. Going to dissect this later today or tomorrow. Based on my knowledge and and this information, my guess is that there is a NAT or some network configuration on this wifi that is breaking safari and iOS. I will do more testing as soon as I get a chance.

[RamyaAshika]

Hi @abardik

I have tried to connect with one peer to another peer. getusermedia works on one peer and I'm sharing the stream using MediaStream API. //Making call if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) { navigator.mediaDevices.getUserMedia({ audio: false , video: true }).then(function (mediaStream) { conn = peer.call(destId, mediaStream); myvideoplay.srcObject = mediaStream; myvideoplay.addEventListener('loadeddata', function () { myvideoplay.play(); // start playing update(); //Start rendering }); }, function (err) { alert(err); }); }

//Receiving call peer.on('call', function (call) { if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) { navigator.mediaDevices.getUserMedia({ audio: false , video: true }).then(function (mediaStream) { call.answer(mediaStream); call.on('stream', function (mediaStream) { myvideo1.srcObject = mediaStream; myvideo1.addEventListener('loadeddata', function () { myvideo1.play(); // start playing update(); //Start rendering }); }); }, function () { alert("Error! Make sure to click allow when asked for permission by the browser"); }); } else { console.log('browser wont support') } });

but its returning as a black screen in ios. Please Help needed.

[Bobisback]

@RamyaAshika Ya I am not sure if it related to what you are doing, but in my case ios 10+ has 100% confirmed to break navigator.mediaDevices.getUserMedia access on embedded web browsers. With no plans to support it in the future. You can google and find lots of information on the subject. it works in safari but not on embedded iOS devices aka MKWebView. Based on my research i have found some workarounds. The one I am trying now is through AudioInput and AudioNode. More specifically this link right here was the magic for me: https://github.com/edimuj/cordova-plugin-audioinput/issues/64

Aka safari mobile navigator.mediaDevices.getUserMedia works, but if you use an embedded webview aka MKWebView what cordova and basically every other hybrid mobile app is built in navigator.mediaDevices.getUserMedia just flat does not work.

[abardik]

@RamyaAshika, Are you sure your

var video = document.querySelector('#your_video_id');
video.setAttribute('autoplay', 'autoplay');
video.setAttribute('playsinline', 'playsinline');
video.setAttribute('webkit-playsinline', 'webkit-playsinline');

But if it doesn't help, just bring back the

[abardik]

@Bobisback, I don't remember exactly on what platform mediaDevices.getUserMedia doesn't work (maybe, as you said, it's cordova webview), but old style navigator.getUserMedia works very well. So, I use this:

window.getUserMedia = function(constraints, onstream, onerror) {
    if ( navigator.mediaDevices ) {
        navigator.mediaDevices.getUserMedia(constraints).then(onstream).catch(onerror);
    }
    else {
        var _getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia;
        _getUserMedia(constraints, onstream, onerror);
    }
}

Works on all platforms (ios cordova webview, ios safari, mac safari, android cordova webview, android chrome, windows chrome, windows firefox, windows electron, mac electron).

[abardik]

@RamyaAshika, And for iOS phonegap/cordova app you have to add this in your config.xml to allow video to play automatically without user action:

<platform name="ios">
    <preference name="MediaPlaybackRequiresUserAction" value="false"/>
    <preference name="mediaTypesRequiringUserActionForPlayback" value="0"/>
    <preference name="AllowInlineMediaPlayback" value="true"/>
</platform>

[RamyaAshika]

Okay fine I ll try and let you know @abardik and one more thing in my IOS mobile its showing error message like "The current browser does not support webRTC" like that its showing. Please whether it will work for all version ?? @abardik

[Bobisback]

@Bobisback, I don't remember exactly on what platform mediaDevices.getUserMedia doesn't work (maybe, as you said, it's cordova webview), but old style navigator.getUserMedia works very well. So, I use this:

window.getUserMedia = function(constraints, onstream, onerror) {
  if ( navigator.mediaDevices ) {
      navigator.mediaDevices.getUserMedia(constraints).then(onstream).catch(onerror);
  }
  else {
      var _getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia;
      _getUserMedia(constraints, onstream, onerror);
  }
}

Works on all platforms (ios cordova webview, ios safari, mac safari, android cordova webview, android chrome, windows chrome, windows firefox, windows electron, mac electron).

Based on my testing and research, on old versions of iOS this worked. On iOS 10+ both of these failed to work. Though I was not aware of the autoplay for iOS so that might have been the issue for me. I will test with it. When I tested it do matter what version of getUserMedia I used it was null on iOS cordova. So I do not believe it is related to the autoplay you speak off.

Thanks, Bob

[abardik]

@RamyaAshika, WebRTC in Safari is supported started from iOS 9, I beleive. If you use any other browser, like Chrome or Firefox, I don't know do they support WebRTC or not, but Safari does.

[abardik]

@Bobisback, Yes, of course, WebRTC is absent in any cordova webview. I proceeded from the fact that you've already installed iosrtc plugin, as I mentioned earlier:

https://github.com/BasqueVoIPMafia/cordova-plugin-iosrtc

and initialized it:

document.addEventListener('deviceready', function () {
  if ( window.device.platform === 'iOS' && cordova.plugins.iosrtc ) {
    cordova.plugins.iosrtc.registerGlobals();
  }
}

Then, you can use all of the following:

navigator.getUserMedia
navigator.webkitGetUserMedia
navigator.mediaDevices.getUserMedia
navigator.mediaDevices.enumerateDevices
window.RTCPeerConnection
window.webkitRTCPeerConnection
window.RTCSessionDescription
window.RTCIceCandidate
window.MediaStream
window.webkitMediaStream
window.MediaStreamTrack

And be sure you read this:

https://github.com/BasqueVoIPMafia/cordova-plugin-iosrtc/blob/master/docs/iosrtc.md

and this:

https://github.com/BasqueVoIPMafia/cordova-plugin-iosrtc/blob/master/docs/videoCSS.md

[RamyaAshika]

Many many thanks I 'll try and let you know @Bobis But I tried in one of my iPhone it's thrown a alert like it's not supported by webRTC I donno why it happened and as well as autoplay is not working.

On Thu, 13 Dec 2018 10:55 pm Andy Bard, notifications@github.com wrote:

@RamyaAshika https://github.com/RamyaAshika, WebRTC in Safari is supported started from iOS 9, I beleive. If you use any other browser, like Chrome or Firefox, I don't know do they support WebRTC or not, but Safari does.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/peers/peerjs/issues/347#issuecomment-447051779, or mute the thread https://github.com/notifications/unsubscribe-auth/AkxCQtfIdLoRoKaWIeULrEcx9tITMRK8ks5u4o2GgaJpZM4IkV8R .

[RamyaAshika]

On IOS10 it's not working @abardik Its showing error message like WEBRTC is not supported by the browser like that its showing.

[RamyaAshika]

@Bobisback IOS 12 it is working but on IOS 10 it's not at all working what to do. At least I need to make this from IOS 9 @abardik.Please help needed.

[abardik]

@RamyaAshika, yes, sorry, it's included to Safari in iOS 11. Before that you can use webrtc in native apps only. For example, you can build cordova app and include iosrtc plugin to support WebRTC, but not in Safari.

[RamyaAshika]

Oops !! Whether it's a different procedure or else the same steps I need to do? Because I'm building ionic 3 angular 5 app. From my app only I'm opening this link in external browser's. Can I able to move further by using this?

Thanks RAMYA D

On Fri, 14 Dec 2018 10:30 pm Andy Bard, notifications@github.com wrote:

@RamyaAshika https://github.com/RamyaAshika, yes, sorry, it's included to Safari in iOS 11. Before that you can use webrtc in native apps only. For example, you can build cordova app and include iosrtc plugin to support WebRTC, but not in Safari.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/peers/peerjs/issues/347#issuecomment-447386801, or mute the thread https://github.com/notifications/unsubscribe-auth/AkxCQqRKlycXzPYkecpa6uguCXTRew_qks5u49k7gaJpZM4IkV8R .

[abardik]

@RamyaAshika, As I know, Ionic doesn't provide its own webrtc support. You have to add iosrtc plugin to your app (see my comments above). Then you can use WebRTC API with some restrictions, which you can find in iosrtc plugin docs.

[RamyaAshika]

Okay I 'll try and let you know. Thanks @Andy Bard

On Fri, 14 Dec 2018 11:24 pm Andy Bard, notifications@github.com wrote:

@RamyaAshika https://github.com/RamyaAshika, As I know, Ionic doesn't provide its own webrtc support. You have to add iosrtc plugin to your app (see my comments above). Then you can use WebRTC API with some restrictions, which you can find in iosrtc plugin docs.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/peers/peerjs/issues/347#issuecomment-447402070, or mute the thread https://github.com/notifications/unsubscribe-auth/AkxCQo-sGa-ayUvogQi-DmmzWdScmsPoks5u4-XDgaJpZM4IkV8R .

[Bobisback]

@abardik So based on the information posted here, if you want to use webRTC with cordova, it is not possible to use Peerjs?

If you use iosrtc then peerjs cannot work correctly right? So the options are to roll my own signalling using iosrtc or not supporting ios?

I am just confused about how iosrtc can work along side peerjs, guessing they cannot work correctly together? Or if I have both libraries installed iosrtc just magically overrides the functions that peerjs use? Just a bit confused how this fits togeth.

I am also trying to build an ionic project using cordova. Again I got it working with peerjs, but similar to the data connection if I call to the ios device, I get a voice connection one way but not the other, if I call from the ios device it seems to work fine with audio going both ways?

I am getting the stream through audio input, so I know 100% that the audio stream is there. Also I do not need video so, just need audio.

Any thoughts?

P.S where does the below code go?

document.addEventListener('deviceready', function () {
  if ( window.device.platform === 'iOS' && cordova.plugins.iosrtc ) {
    cordova.plugins.iosrtc.registerGlobals();
  }
}

I am guessing in the device ready equivalent for ionic. If so then my next question is how do I import it correctly.

[abardik]

@Bobisback, peerjs works perfectly together with iosrtc. iosrtc plugin includes libwebrtc library to your cordova app and implements WebRTC API, which is absent in cordova webview. Peerjs uses WebRTC API. Without iosrtc.registerGlobals you have to write different code for browsers and for cordova app. With registersGlobals all WebRTC API provided by iosrtc plugin is registered globally (and it's the same as WebRTC API in any browser), so peerjs can use it like in any browser, and your code is the same for all platforms. Please, read iosrtc and peerjs docs.

[abardik]

I don't know the details of ionic framework, but if I understand right, it runs inside cordova app. registerGlobals should be called in cordova's deviceready event handler (you can place this code anywhere in your JS; it will be executed after cordova is initialized and all plugins are loaded).

[RamyaAshika]

Ok fine I want to pass the stream to peer connection without enabling their camera. In one html peer.on('connection', function (conn1) { conn1.on('data', function (data) {
var conn = peer.connect(data); conn.send("from desktop"); conn.on('open', function (id) { const sUsrAg = navigator.userAgent;
//making call to pass annotations stream to receiver var captureStream = document.getElementById('main-canvas').captureStream(); conn = peer.call(data, captureStream);
}); }); });

This is how I'm passing the stream to the other user. In another html peer.on('call', function (call) { if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) { navigator.mediaDevices.getUserMedia({ audio: false , video: true }).then(function (mediaStream) { call.answer(mediaStream); }, function (err) { //alert("Error! Make sure to click allow when asked for permission by the browser"); alert(err); }); } else { console.log('browser wont support') } });

In this I don't want to enable their camera,I just want to get the remote stream from one html. Is that possible? @abardik @Bobisback

[Bobisback]

@abardik Thanks for the comments, I am still trying to get iosrtc installed atm. Getting some errors related to swift and I cannot seem to figure out the issues with it. Once I get this sorted I will move back to trying to get peerjs to work.

If you have any ideas here is the thread: https://github.com/BasqueVoIPMafia/cordova-plugin-iosrtc/issues/338

@RamyaAshika If you are trying to just send a stream from one peer to another with nothing going the other direction, peerjs docs says this is possible. Here is the documentation on the subject.

When calling or answering a call, a MediaStream should be provided. The MediaStream represents the local video (webcam) or audio stream and can be obtained with some (browser-specific) version of navigator.getUserMedia . When answering a call, the MediaStream is optional and if none is provided then a one-way call is established. Once the call is established, its open property is set to true.

I cannot comment on your code, since I decided not to use the get media devices function. I only needed audio so I decided to go with using audioinput plugin to get the audio stream.

[RamyaAshika]

ooh, k What I'm really trying to do is,

I had made this successfully on chrome, firefox and so on by making two media connections I can able to share the video as well as on top of that I can able to pass canvas as a media stream(my drawings on top of the video)but now it's not working in safari.

When we making two media connections using getUserMedia(). In safari disables the first connection and show the black screen instead of apply on top. I want to pass my canvas stream from peer.call() without getUserMedia.Please help needed. Is that possible? Because getUserMedia is the one asking permission for the camera.

peer.on('call', function (call) { call.answer(stream); //results as ReferenceError: stream is not defined }

because without getUserMedia it shows like this but what I want to do is I want to pass only video streams to my video tag @Bobisback

[Bobisback]

@RamyaAshika I am not sure what your "canvas stream" is. When you answer the call you can pass in null or you need to pass in a mediastream object. I am still not 100% sure what you are trying to do. If you are just trying to get the video stream without audio, then I do not see why you cannot just pass in the options saying no audio. Seems like the getUserMedia should be fine for you to use?

@abardik So I managed to get iosRTC working, I think, not sure how to tell tbh... ahaha. I am still having issues though. I seem to be having an issue with playback on iOS though. Based on the logs I am getting the streams correctly, and the connections are working, but no audio is coming through on iOS.

Also if I connect two apple devices together, it does the same thing as the data connection. It acts like it is connecting but nothing happens. Does not appear to get the streams either.

[abardik]

@RamyaAshika, ReferenceError: stream is not defined - it's just because you have no stream variable defined, not a media stream itself. Check your variables names (in your previous example it was called mediaStream, not stream). And about video-only streams. Of course, you can use them as @Bobisback mentioned above, calling getUserMedia with { video: true, audio: false }.

[abardik]

@Bobisback I never had any problem with iosrtc (or any other plugin) installation because I use Phonegap Build to build my projects, and every well-developed plugin is compatible with Phonegap Build. So, just to make things simple I refused to learn and use xcode, native SDKs, etc. But that's great if you figured it out on xcode. About your problem. Did you try a simple webrtc example without peerjs? You have to get a pure webrtc working first. Then you can proceed to peerjs with a 100% guarantee that your network configuration is not an issue.

[abardik]

@Bobisback One time I had a problem with Chrome-Safari webrtc streams because of different media codecs available in these two browsers. Safari has H.264, Chrome has VP8 and H.264, and it uses VP8 by default. When you establish a direct p2p webrtc connection both browsers can negotiate about codecs, and everything works well. But in my case it was intermediate SIP server (freeswitch), to which both browsers were connected to. All traffic was going through this server mixed together and converted to a single codec. The configuration of that server allows to specify what codec to convert to, and if you choose VP8, Safari shows black screen instead of video. The same with audio codecs. They are different on different browsers. But if browsers are connecting directly (as peerjs allows us to do) they can negotiate what codecs to use. The same thing with cordova app. I had green (not black) screen until I enabled both codecs on freeswitch. So, there are many reasons why you can get a black screen. Try to start from a simplest webrtc example to confirm that it's a peerjs issue, not a network's or browser's one.

[Bobisback]

@abardik Thanks for the input, I will try to get a simple webrtc application going next. I am only trying to send audio, I do not care about video at all.

Do you happen to have a simple sample project for phonegap that uses just WebRTC, or maybe some guides?

I need to figure out quickly if webrtc is a good solution, if I cannot get this to work on iOS then I need to move on to greener pastures.

Just need to figure what to install. The basics of webrtc is just opening a connection with each other. I will look at the docs and see if I can figure it out.

[abardik]

@Bobisback https://github.com/muaz-khan/WebRTC-Experiment You will find a lot of examples over there. WebRTC is a good solution, I assure you. It works fine on every OS and almost every browser.

[RamyaAshika]

Actually this is a bug in safari "New getUserMedia() request kills existing stream track" https://webrtchacks.com/guide-to-safari-webrtc/ but I want to get the stream fron peer.call without getUserMedia. Is it possible? or else I want to pass the stream from one html to another html. Any idea,Please help needed @abardik

[abardik]

@RamyaAshika I can confirm that bug. I workaround it with a global audio stream captured when the call is started and switching video streams from front and back camera (in the following getUserMedia calls with video only constraints) and adding/removing them to/from PeerConnection with addTrack/removeTrack. It works fine. But yes, if you try to getUserMedia a media source (for example, microphone) that you already captured in previous getUserMedia call and didn't release it yet - you'll receive an error that media source is already captured. I'm not sure, if you can use MediaStream from canvas in cordova app (because iosrtc plugin has its own non-standard and non-totally-compatible implementation of MediaStream). But I think you can do this in Safari, because it has a pure standard MediaStream. So, yes you can try to grab MediaStream from one canvas, add it to PeerConnection, and then attach it to another canvas on the remote peer. I never tried canvas streams, but theoretically it sounds good.

[Bobisback]

@abardik Sorry I know I am bugging you a ton, but I really wanna get this working. I was looking through the samples for the webRTC. Right now my objective is to get a working connection between iOS and Android devices using peer to peer audio, at first I wanna get it working in browsers. That's all I need. I am very unfamiliar with WebRTC, it even took me a second to grasp PeerJs and compared to WebRTC Peerjs seems super simple. Could you point me in the right direction on what sample I should focus on to get basic peer to peer audio going, ideally without a server. I am just not sure what all is needed for WebRTC to work, ideally I want the simplest solution, if that means manually typing ips in that is fine. If I can get away from using an server that is good as well.

Once I get it working in the browser, then I will move on to using it in cordova. The cool part is ionic is built on top of phonegap so it should still work in ionic. There is a lot of pieces to this pizzule and my brain is exploding trying to figure out how to fit them all together. Now trying to learn WebRTC is making it more complicated. Any pointers on where to start would be great.

As far as getting iosrtc working I had to modify the xcode project as iosrtc does not work with the latest version of xcode. I am going to try and downgrade my version I am using to see if it makes it easier.

Thanks, Bob

[abardik]

@Bobisback,

https://www.html5rocks.com/en/tutorials/webrtc/basics/

I think, you can start from this very short but powerfull tutorial with live examples inside. You will start from getUserMedia and AudioContext, then go to local peer-to-peer video chat (in one browser tab, but you will learn a lot about internals of webrtc), and finally you'll try their fully functional video chat with their own signaling server (analogue of peerjs server).

To be sure that webrtc is good enough for your project you can just load https://appr.tc in any browser, including iOS Safari and Android Chrome, and you will see that everything just works.

After that you can proceed with peerjs. peerjs is not only client wrapper for webrtc API. There is a server side of peerjs, which does exactly what appr.tc server does - transfers SDP and candidates between peers (via WebSockets). Without signaling server your users should exchange SDP and candidates manually (for example, via skype, because SDP is just a text). But, of course, it's not an option. So, in your final app you have to implement not only client GUI, but install and maintain your own copy of peerjs server (you can do it on your own windows or linux machine inside your corporate network or you can rent a minimal AWS virtual machine with nodejs installed).

Anyway, I suggest to start from that tutorial. You will learn enough to proceed next by yourself.

Good luck!

[Bobisback]

Thanks for the link, I will check it out for sure. I found another really good tutorial here as well. It is not as complciated as I imagined. Just trying to get the signalling separated from the actual webRTC stuff. I am not actually 100% on what part is signalling and what is not, and what part is client and what part is server. Still going through the tutorials so should have it figured out shortly.

Based on what I have read and seen (besides iosrtc to make ios work) I do not need to actually install anything to use the WebRTC stuff, aka there are no npm install or package.json things I need to add? It just works out of the box with no imports or anything? If that is the case I just need to figure out how to use it with typescript, guessing just define a var somewhere and use like normal.

As far as https://appr.tc/ goes, that is really cool, but I am not sure it applies to me. How can I go to that on an iOS or android device in cordova? That is really what I am trying to test is cordova, not the browsers. The issue is iOS safari works but cordova does not use iOS safari it uses MKWebView or the other one SFSafariViewController or something and those two do not work with WebRTC, my understanding is that iosrtc fixes this. So is there a way to setup iosrtc and a basic cordova app and just direct link the https://appr.tc/ website in the code? Would this even work? Guessing not.

So my goal right now is to create a basic cordova project with iosrtc and WebRTC and get some basic communication between iOS devices.

One other issue I run into a lot is when I get an audio stream from a remote host in android, I get a error related to origin not accepted. Any ideas what this is?

[abardik]

@Bobisback Signaling is a transfer of SDP (offer and answer) and candidates between peers. It should be done with a help of signaling server (which is peerjs server is) or manually by users (for example, via any IM). WebRTC can not transfer SDP and candidates by itself. WebRTC API is only this: https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API

Yes, you do not have to install anything if you will use a plain WebRTC API and your users will exchange their SDPs and candidates manually. But if you want to simplify your development process and your users' lives, you have to use some existing library to abstract from WebRTC API and the most important - from signaling part. That's why peerjs is needed. With peerjs you can forget about candidates, SDP and other internal things of WebRTC. You just create data and media connections, and peerjs will do the rest, including a creation and a transfer of SDP and candidates.

Regarding npm install, package.json, etc. For client side you don't have to install anything. You can just include <script src="peer.js"> in your html. For server side you should install nodejs, npm and run your own peerjs server.

There is no cordova webview that has WebRTC support. You can bring WebRTC to cordova webview with the help of iosrtc plugin as discussed earlier.

Yes, you can redirect your cordova app directly to any URL. This is simpler than you think. Just create a Phonegap Build app (https://build.phonegap.com) and set <content src="https://appr.tc"/> in your config.xml. Of course, you should add iosrtc and other plugins to your config.xml (for example, plugins to ask user permissions to use camera and microphone, whitelist plugin to access different URLs, and others). But it's another story. But yes, you can do it, and it's the simplest way to build what you want to build, as I understood.

All 'origin' issues (if we are speaking about cordova) can and should be solved with whitelist plugin (https://cordova.apache.org/docs/en/latest/reference/cordova-plugin-whitelist/), which allows you to allow your cordova app to access some (or all) remote resources.

[Bobisback]

So I created a new project to get this working. It would seem that iosrtc is not compatible with the newest changes to the webRTC library?

example, the old function was called RTCPeerConnection.addStream, new one is called RTCPeerConnection.addTrack.

With my current code base on ios I am getting errors related to this.

TypeError: _this.pc.addTrack is not a function. (In '_this.pc.addTrack(track, _this.localStream)', '_this.pc.addTrack' is undefined)

Any ideas how to get a around this?

[abardik]

@Bobisback That's right. Add your MediaTrack to MediaStream and use addStream for ios cordova app. You can do this on all your clients, because the old API is still supported.

[abardik]

@Bobisback Even if the old API will be totally removed from all browsers, you still can use it in your cordova project, because you can choose what versions of plugins to use. The current versions of WKWebView (1.1.4) and iosrts (4.0.2) both support addStream. For the rest of clients I suggest to use addTrack or to write a shim.

[Bobisback]

@abardik Thanks for the input. So I got WebRTC working now in a test project with ionic 4. So this time if I do browser to device it works fine. i get audio transfer on both with no issues. Connecting looks fine in the logs. it appears to work both ways. But if I connect iOS to iOS, all the logs look great. But I just get no audio playing. I have added the auto play stuff to the info.plist, and since the audio works fine on the browser connected to iOS, this means at least I know I am getting mic and the speakers work. I am not sure what is going on. Any ideas why iOS to iOS would just have no audio, should I start looking at codex's I guess?

Frist iOS Device log

[Log] Starting to get stream audioinput (cordova.js, line 1732)
[Log] Got stream audioinput (cordova.js, line 1732)
[Log] Message from client: Asking to join room Local_Channel (cordova.js, line 1732)
[Log] Adding local stream. (cordova.js, line 1732)
[Log] Received request to create or join room Local_Channel (cordova.js, line 1732)
[Log] Room Local_Channel now has 2 client(s) (cordova.js, line 1732)
[Log] Client ID CwNYTbbYTkSVg2fHAAAA joined room Local_Channel (cordova.js, line 1732)
[Log] Client Id: CwNYTbbYTkSVg2fHAAAA Joined Local_Channel (cordova.js, line 1732)
[Log] Client said: got user media (cordova.js, line 1732)
[Log] Client received message: – {sdp: "v=0
↵o=- 4517823112078004418 2 IN IP4 127.0.0.1
↵s…6661 label:e2154dae-c6e7-459c-8cb4-bbfef633aaea
↵", type: "offer"} (cordova.js, line 1732)
{sdp: "v=0
↵o=- 4517823112078004418 2 IN IP4 127.0.0.1
↵s…6661 label:e2154dae-c6e7-459c-8cb4-bbfef633aaea
↵", type: "offer"}Object
[Log] >>>>>>> maybeStart()  – false (2) (cordova.js, line 1732)
MediaStream {id: "1e85cd93-bedf-48b4-9ee7-023db6591fdc", active: true, onaddtrack: null, onremovetrack: null, getAudioTracks: function, …}
true
[Log] >>>>>> creating peer connection (cordova.js, line 1732)
[Log] Created RTCPeerConnnection (cordova.js, line 1732)
[Log] isInitiator – false (cordova.js, line 1732)
[Log] Sending answer to peer. (cordova.js, line 1732)
[Log] Remote stream added. (cordova.js, line 1732)
[Log] setLocalAndSendMessage sending message – {sdp: "v=0
↵o=- 8077396569099575851 2 IN IP4 127.0.0.1
↵s…0037 label:80147144-60ac-4e09-981c-ec7beaf19a4c
↵", type: "answer"} (cordova.js, line 1732)
{sdp: "v=0
↵o=- 8077396569099575851 2 IN IP4 127.0.0.1
↵s…0037 label:80147144-60ac-4e09-981c-ec7beaf19a4c
↵", type: "answer"}Objectsdp: "v=0
↵o=- 8077396569099575851 2 IN IP4 127.0.0.1
↵s=-
↵t=0 0
↵a=group:BUNDLE audio
↵a=msid-semantic: WM…"type: "answer"Object Prototype__defineGetter__(propertyName, getterFunction)__defineSetter__(propertyName, setterFunction)__lookupGetter__(propertyName)__lookupSetter__(propertyName)constructor: function()hasOwnProperty(propertyName)isPrototypeOf(property)propertyIsEnumerable(propertyName)toLocaleString()toString()valueOf()
[Log] Client said: [object Object] (cordova.js, line 1732)
[Log] icecandidate event:  – RTCPeerConnectionIceEvent {isTrusted: true, candidate: null, url: null, …} (cordova.js, line 1732)
RTCPeerConnectionIceEvent {isTrusted: true, candidate: null, url: null, type: "icecandidate", target: RTCPeerConnection, …}RTCPeerConnectionIceEvent
[Log] End of candidates. (cordova.js, line 1732)

Second ios device log

[Log] Starting to get stream audioinput (cordova.js, line 1732)
[Log] Got stream audioinput (cordova.js, line 1732)
[Log] Message from client: Asking to join room Local_Channel (cordova.js, line 1732)
[Log] Adding local stream. (cordova.js, line 1732)
[Log] Received request to create or join room Local_Channel (cordova.js, line 1732)
[Log] Room Local_Channel now has 1 client(s) (cordova.js, line 1732)
[Log] Client ID N4sWw5nhUaC0UN37AAAJ created room Local_Channel (cordova.js, line 1732)
[Log] Client Id: N4sWw5nhUaC0UN37AAAJ Created Local_Channel (cordova.js, line 1732)
[Log] Client said: got user media (cordova.js, line 1732)
[Log] Another peer made a request to join room Local_Channel (cordova.js, line 1732)
[Log] This peer is the initiator of room Local_Channel! (cordova.js, line 1732)
[Log] Client received message: – "got user media" (cordova.js, line 1732)
[Log] >>>>>>> maybeStart()  – false (2) (cordova.js, line 1732)
MediaStream {id: "2800710b-ba0b-440d-b451-21f8274be567", active: true, onaddtrack: null, onremovetrack: null, getAudioTracks: function, …}
true
[Log] >>>>>> creating peer connection (cordova.js, line 1732)
[Log] Created RTCPeerConnnection (cordova.js, line 1732)
[Log] isInitiator – true (cordova.js, line 1732)
[Log] Sending offer to peer (cordova.js, line 1732)
[Log] setLocalAndSendMessage sending message – {sdp: "v=0
↵o=- 4517823112078004418 2 IN IP4 127.0.0.1
↵s…6661 label:e2154dae-c6e7-459c-8cb4-bbfef633aaea
↵", type: "offer"} (cordova.js, line 1732)
{sdp: "v=0
↵o=- 4517823112078004418 2 IN IP4 127.0.0.1
↵s…6661 label:e2154dae-c6e7-459c-8cb4-bbfef633aaea
↵", type: "offer"}Object
[Log] Client said: [object Object] (cordova.js, line 1732)
[Log] Client received message: – {sdp: "v=0
↵o=- 8077396569099575851 2 IN IP4 127.0.0.1
↵s…0037 label:80147144-60ac-4e09-981c-ec7beaf19a4c
↵", type: "answer"} (cordova.js, line 1732)
{sdp: "v=0
↵o=- 8077396569099575851 2 IN IP4 127.0.0.1
↵s…0037 label:80147144-60ac-4e09-981c-ec7beaf19a4c
↵", type: "answer"}Object
[Log] Remote stream added. (cordova.js, line 1732)
[Log] icecandidate event:  – RTCPeerConnectionIceEvent {isTrusted: true, candidate: null, url: null, …} (cordova.js, line 1732)
RTCPeerConnectionIceEvent {isTrusted: true, candidate: null, url: null, type: "icecandidate", target: RTCPeerConnection, …}RTCPeerConnectionIceEvent
[Log] End of candidates. (cordova.js, line 1732)

[Bobisback]

Do I need to manually install adapter.js to my cordova project? if so how do you do that?

maybe that is what I am missing.

[abardik]

@Bobisback I don't think you need adapter.js, but you can add it for better compatibility with other browsers. My project works the same with and without it. Just add it to your app with Githubissues.

Githubissues is a development platform for aggregating issues.