WebRTC Support for VoIP calls

[tmolitor-stud-tu]

Well, I did some research and found this good documentation on webrtc on iOS: https://tech.appear.in/2015/05/25/Getting-started-with-WebRTC-on-iOS/ and: https://www.html5rocks.com/en/tutorials/webrtc/infrastructure/

for signalling jingle could be used (this is used by monal, too)...for an implementation combining jingle and webrtc you could port this javascript library: https://github.com/ESTOS/strophe.jingle

A list of other javascript jingle/webrtc implementations can be found here: https://github.com/legastero/jingle-interop-demos#jingle-interop-demos

The culprit in using jingle for signalling seems to be the translation between sdp (used by webrtc) and jingle messages...a java implementation could be found here (but reading/porting the above javascript implementation should give the exact same results): https://github.com/tuenti/sdp-to-jingle-java

jingle rtp sessions are defined in XEP-0167: https://xmpp.org/extensions/xep-0167.html

if you want to provide a native look and feel for (incoming) calls you could also use this: https://developer.apple.com/reference/callkit

I hope this info is helpful. If you give me a starting point by implementing a basic webrtc gui, I can work out the rest (sdp to jingle translation and jingle session negotiation) Is some sort of jingle (maybe for filetransfer) already implemented in chatsecure, btw?

[tmolitor-stud-tu]

We could also not implement jingle and do something different instead: just use a new URI scheme like webrtc:// for signalling (like the aesgcm:// scheme used for encrypted http uploads).

This would be more straightforward when OMEMO/OTR encryption is involved, because sending such data in a normal message encrypted by OMEMO would automatically protect the key used for media encryption.

XEP-0167 mentions the possibility to use an \<encryption/> element inside the jingle session initialisation iq, but this would not be automatically encrypted by OMEMO/OTR and thus be leaked to the xmpp server (breaking forward secrecy etc.).

Reviewing our options, I think we should go with the webrtc:// thing which would be easier to implement and be more secure at the same time.

[chrisballinger]

I'd like to avoid all of the legacy baggage of jingle/sip/rtp/xmpp and if possible bootstrap everything via a single message sent thru OMEMO. Using a webrtc:// scheme seems like a good idea.

On Wed, Apr 12, 2017 at 4:05 PM, tmolitor-stud-tu notifications@github.com wrote:

We could also use something different and just use a new URI scheme like webrtc:// for signalling (like the aesgcm:// scheme used for encrypted http uploads).

This would be more straightforward when OMEMO/OTR encryption is involved, because sending such data in a normal message encrypted by OMEMO would automatically protect the key used for media encryption.

XEP-0167 mentions the possibility to use an element inside the jingle session initialisation iq, but this would not be automatically encrypted by OMEMO/OTR and thus be leaked to the xmpp server (breaking forward secrecy etc.).

Reviewing our options, I think we should go with the webrtc:// thing which would be easier to implement and be more secure at the same time.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ChatSecure/ChatSecure-iOS/issues/738#issuecomment-293731871, or mute the thread https://github.com/notifications/unsubscribe-auth/AAfqH0g7-jySvpRjMTKwjJXhAEGU8VEaks5rvVi3gaJpZM4M8EVM .

[chrisballinger]

Here's more info on how Signal-iOS builds WebRTC: https://github.com/WhisperSystems/Signal-iOS/blob/master/BUILDING.md#building-webrtc

[tmolitor-stud-tu]

Well, I'll try to build it that way. Where should I put the resulting WebRTC.framework file inside the chatsecure directory structure?

I'm not really good at developing a gui for iOS, it would be nice if you could implement just a simple "call me"-button displayed somewhere...I'll add the webrtc:// logic afterwads :)

[afriedmanGlacier]

Good stuff, we are interested in seeing this completed too

[tmolitor-stud-tu]

@afriedmanGlacier It would be completed faster, if you could donate some developer time for this.

My own time is fairly limited :(

[chrisballinger]

Don't directly checkin the framework to the repo. Once I merge the xep-0363 branch there will be an area for Carthage builds that isn't checked in, and it could be stored there.

From what I understand this doesnt require us to run any services if we use Google's STUN/TURN server?

[tmolitor-stud-tu]

I don't know if google operates a free TURN server.

STUN is available here, though:

• stun.l.google.com:19302
• stun1.l.google.com:19302
• stun2.l.google.com:19302
• stun3.l.google.com:19302
• stun4.l.google.com:19302

[tmolitor-stud-tu]

But I'm willing to donate a complete and exclusive vServer running Debian Jessie, if that helps :)

[chrisballinger]

Oh ok if we only need STUN then hopefully Google will suffice. Thanks for your help!

[tmolitor-stud-tu]

In many/some? scenarios you need a TURN server because of symmetric NATs.

[tmolitor-stud-tu]

Okay, I did some research and found that symmetric NATs seem to be very rare. I also confirmed that the NAT of my mobile carrier is not symmetric, too.

--> Using google's STUN servers will suffice in the most scenarios, imho :)

[tmolitor-stud-tu]

Okay, apparently my tests had errors. At least my mobile carrier in germany uses symmetric NAT for their carrier grade IPv4 NAT (and I suspect other mobile carriers to do the same).

Symmetric NAT seems to be rarely seen on landline connections, but to be common for mobile carriers. --> We definitely need a TURN server if we want to support VoIP calls using a mobile data plan instead of WiFi.

[afriedmanGlacier]

@tmolitor-stud-tu unfortunately my time is fairly limited too. At the moment I am trying to figure out the OMEMO issues that we are seeing using ChatSecure on eJabberd. This is a showstopper for us. Once that gets figured out, I have a few other things I need to do first. If it still isn't done when my other mandatory tasks are done I can probably work on it.

[tmolitor-stud-tu]

@afriedmanGlacier Well, better than nothing I guess :)

@chrisballinger I did some more research on the signalling part: I found this serverless WebRTC videochat project: http://blog.printf.net/articles/2013/05/17/webrtc-without-a-signaling-server/ https://github.com/cjb/serverless-webrtc (just for reference: https://github.com/wojta/serverless-webrtc-android)

The signalling part is done by copy 'n paste of text snippets containing the SDP offer and response. These offer snippet is about 6 KiB in size, that's really big for an xmpp message. The serverless application linked above uses a data channel which consumes ~1 KiB in the sdp offer and is not needed for us. The SDP response is smaller (only ~2 KiB in size).

I tried to compress the offer with gzip --best, but this still gives us about 1.4 KiB size. Urlencoding this compressed data gives us ~3.4 KiB. Base64 encoding the compressed data gives us only ~1.8 KiB. Urlencoding the Base64 encoded compressed data gives us ~ 2 KiB. --> I'd go with compressing, base64 encoding and ulencoding the SDP offer and response.

Sidenote 1: My laptop has more IP addresses than a normal mobile phone, that could reduce the size of the encoded and compressed SDP offer to ~ 1.5 KiB on a mobile phone.

Sidenote 2: Using another compression algorithm could reduce the size further (7zip etc.)

Example webrtc URI which would be sent as xmpp message: webrtc://offer=eNq1mG1v2zYQx9%2FvUwh5OUz28Xh8HPzCjputQFsYTdq9KVDQEtUYjR9gKWm2IN99J9tLrLRWFk9DgkQSeUfyxP%2Fdj7o7qf5cxRN%2FsiyKuD755aTMV3x3M4BP60%2BL5WC%2B%2FGt2dRV6vd7F76%2FPP%2FPv%2BXgyfH8xTEn1bA8StEBOkwQprUOtnYYEktfvktcTSqC3%2BaldlYO0%2FlcNuLm%2BCIMyLvJ1zG62d8Vs8SWuV%2BvZovLlZUhR6YSUH5I%2FUx5Gfjj2duTHQ69OPY796ZkfnnplvB75s5EX6M%2BMd8oL8q9O%2FdnYj7RX4M%2BkH8q61Y09aT%2Fiv6PaD1qPZjvwl%2FXyeuVHH96N37xKePVhXYXP8HAlHq5w23%2BWxXS5qmbLRemr9Sz7ehW3DfNylqdlnIdFNcv8H2%2FPk5%2FrhvkgXOezZaJQKJN8GE%2F6F2%2FO%2B%2B8vJv3z4cfJWSLAJY5DZuvO2WAXOeGwJ7TtCdmTW%2FdZWOSzPFTRQyJqPwkK5G5ktWx0343E7zW5XJbVU2vcs1ZSaDKP1gA9kRApoQ%2Ba05651miES4SpzXsWa2NjjThorPfHdkbYx5mTZGtlHcBBa7tnbdCiUo8z52WTITq8bLEXtdoB8TSLHMB7Aun8NBa8Nh2Fd6LgEGoDeNjXfgydY3u38zXNpsqTkoodYvTTehRCaw%2FHU9D%2BvKwBMDtfMpFOYovlXjQRlHJS7SxdpqOXPDVvjTPexmzKodWiJTp7sUVUqEjufKGQPtqi8EVE7bOcN5uUDtXh%2FbW%2FPY2Txm0CLbzXeVBeksq9lUieMOTsy6jDbxz3A215hWh2vn40LaesPeiKM1VTNPhENFoStYgGm6LRT0SjjEFqEQ02RWMbolFOO2wRDTZFg03REEpLLaLBpmioIRoNWqo20WBTNNAiGqlJUZtosCka2yYaA4baRINN0ehH0ZC0raLBpmioTTQOhWgTDTZFg4dFo5wg3SYabIoGWkTDYWaPLaLBpmh0i2ikNu6Jq2Zh5pt0WaQP%2Fsvd49tqHlZeJNfrhZ%2FFqvBcJcO89OtqlV6y%2FS3X8XKdpZvql17Fm3j1WEBX33J%2B4xSt1DGynnQuCgDeT5BbOwXMZG4fe18X6%2FDF68iJVOT5rtrOcv9QrR8LsL%2BzU01xSjbVOUJK01ynNhiXuhBACCGzImb3yR1KDqychlQZp1KueTK1sRBpoYwwMQNd5Hi%2F9buuspXf5oeD1bnuk86vb%2F%2B5W22Cw6V9ubou%2B2QBoI%2BNRpf8ZhD7mxbRaIFkcvr2w6al8dzWz4d7z8tYMbqErFqFcvda6pB7i1uNYZItwjz6O8kzQYkyVRxtJrdg0qBAp0EWHHw1FVOQ91tYuZnlcclS1tL9CFYQOsOU7RhHY4oCaCn3z2CKJN1i%2FBymECdCPBpTHIHrCFNkvbE6whQFsj3jHsQUhUZDR5gijZHQDabwBnddYQpxu%2BoGU4iwBcSexxTC%2F4ApXKJBHYspxHhOx2MK5yB9LKYQny1NR5hCQjjsCFMUJwJ1FKbwaUV1hSkEoKAbTGFXQnaEKdJaCR1hisLvNs%2B%2FwpRiXnEd5nnPw21alAMOgbW%2Fbm7WAw3%2FF5GII4hEq2BBFJQGW7iUXAFp0C4ykRR5nhUaIsomkdSpoJ1Iiulm8YuQff3x02R1Nfu%2BJcvmSTFbt6EN9%2Fo4sX0%2Brz9HI2gIrDXamSNwJKx4glmoP7fwAQ9AJuOaSM5PLyY1AXRIIxvnR9OIlOjssTTCOcXY42nEaXE8jWihbFc0whnJdUUjWtnjaESq1mT4Mhpp%2FR71MhpBaPk68TIaUdq1fKF7GY0YKY86%2F3WfM3fHozLbZpha3cm3OOX0k%2FK4IbsMi0W8SlDp9nzDayYgwfvgxfnm5P6nvwHdNJ%2Fw

[chrisballinger]

I think that's plenty small enough for a single XMPP message even uncompressed. We've been abusing XMPP servers for years with OTRDATA's 64kb packets. :)

[tmolitor-stud-tu]

I think compressing them is still better because clients not understanding what this is will show the message unchanged and the bigger the message the worse the user experience is.

I'd add a small explanatory text at the bottom of the message, too. That will make the user experience even better :)

[chrisballinger]

It would be great if OMEMO allowed for "out of experiences" so we could put additional data in the payload...

[tmolitor-stud-tu]

Well, that's right, but currently it doesn't...but we should propose this to Daniel and the others involved with OMEMO...

[tmolitor-stud-tu]

We should add a new <feature> entry to chatsecures disco indicating webrtc support. I'd propose a value of urn:xmpp:webrtc:0. This way clients would only send webrtc:// urls if they know the receipient knows what they mean.

We'd have to write a XEP to be able to use the urn:xmpp namespace, though. But we should do this anyway imho.

[tmolitor-stud-tu]

And the webrtc:// messages should contain the no-copy hint and maybe the no-store hint, too.

[trillerpfeife]

BTW there is a similar approach sending the SDP offer/answer in the element its a Proto XEP called SoX https://xmpp.org/extensions/inbox/sox.html

[kitkatloyalist]

"for every season turn turn turn"

https://github.com/coturn/coturn/blob/master/README.md