Unified plan with Twilio video not working

[it4e]

YOU MUST read first!

Please use Community Forum for general technical discussions and questions.

• [x] I have used Google with the error message or bug in association with the library and Cordova words to make sure the issue I'm reporting is only related to iOSRTC.
• [x] I have provided steps to reproduce (e.g. using sample app code https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample or updated extra/renderer-and-libwebrtc-tests.js file).
• [x] I have provided versions of third party library name, ios, Xcode and plugin version and adapter.js version if used.

Note: If the checkboxes above are not checked (which you do after the issue is posted), the issue will be closed, removing this checkbox will result in automatic closed issue.

Versions affected

• Cordova version (e.g 7.1.0): 10
• Cordova iOS version (e.g 5.1.0): 6.2.0
• Plugin version (e.g 6.0.12): 6.0.20
• iOS version (e.g 10.2): 14, but all versions have this issue
• Xcode version (e.g 11.1 - 11A1027): 12
• Twilio version:: 2.4.0+
• WebRTC-adapter version (e.g. 7.4.0): latest
• WebRTC Framework version (e.g. JSSip 3.1.2): latest

Description

Unified plan SDP is not working with Twilio video. Remote participant audio and video can not be heard and seen, just a blank white screen. Works when connecting with plan-b SDP, but since Twilio is now moving to Unified plan, plan-b will no longer work in the future.

Steps to reproduce

Connect with the exact code in https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample, but removing the

sdpSemantics: 'plan-b',
bundlePolicy: 'max-compat'

parameters to the Twilio connect call. Since Twilio and Google Chrome is removing support for plan-b in August, these parameters will no longer be supported.

The same code was used for both Cordova and web.

Expected results

Be able to see remote video and hear audio with Unified plan.

Actual results

Works as expected when using:

sdpSemantics: 'plan-b',
bundlePolicy: 'max-compat'

But when removing these parameters, it does not work as only a blank screen is present for the remote video and no sound.

See images below:

Cordova:

Web:

[hthetiot]

The information you provided are not enought to reproduce your issue. BE aware that iosrtc works with Twilio for many users, and you issue is most likely due the way you load iosrtc or twilio, see full sample application for reference.

I have provided steps to reproduce (e.g. using sample app code https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample or updated extra/renderer-and-libwebrtc-tests.js file).

You did not even provide the twilio version you are using.

Please reproduce the issue with sample app with twilio.

0. Clone sample https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample

1. Update www/js/index-twilio.js with your TWILIO_TOKEN https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample/blob/master/www/js/index-twilio.js

2. Update index.html to load www/js/index-twilio.js instead of www/js/index-local.js

   • https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample/blob/master/www/index.html#L77

Then test the application.

[it4e]

@hthetiot

Thanks, I have now updated with further information

[hthetiot]

thx @it4e will check and comeback to you.

[hthetiot]

Just curious, have you tried with bundlePolicy: 'max-compat'only but no plan B ? @it4e

Finally, can you try master, master will be release under 8.0.0 and support transceiver and i think the unified plan may have a better implementation and its based on WebRTC M89.

cordova plugin remove cordova-plugin-iosrtc --verbose
cordova plugin add https://github.com/cordova-rtc/cordova-plugin-iosrtc#master --verbose
cordova platform remove ios --no-save
cordova platform add ios --no-save

[it4e]

@hthetiot

Thanks,

When only using bundlePolicy: 'max-compat' it is the same issue.

When using master the sample application freezes and xCode yields the following error:

Maybe related to https://github.com/cordova-rtc/cordova-plugin-iosrtc/issues/652#issuecomment-805877908?

[hthetiot]

Thx @it4e I will investigate when i have a momment.

[hthetiot]

Twilio stupid userAgent snifning instead of features detection, see comments with fix:

• https://github.com/twilio/twilio-video.js/issues/1364#issuecomment-807434214

[hthetiot]

thx to @noahmehl for info.

[it4e]

Thank you tons for checking!

Got it to work on the sample application by applying the patch mentioned in https://github.com/twilio/twilio-video.js/issues/1364#issuecomment-778064625 manually to the CDN file. Another solution is to set the User Agent manually to contain the word Safari, this way we do not have to change anything in Twilio:

<preference name="OverrideUserAgent" value="Safari Cordova" />

Also had to use the IOSRTC master branch to support transceiver and bundlePolicy: 'max-compat' for it to work.

One issue noticed is that in the web-console the following error keeps popping up:

@hthetiot Do you think master is stable enough to use in production as of now?

[hthetiot]

Do you think master is stable enough to use in production as of now?

Yes it should, still its not released until proven as stable as 6+

Also for the getStats issue, please fill separate issue. For instance if you can debug what responseand confirm its RTCStatsReport instance and try this patch to see if that forEach missing from RTCStatsReport.

RTCStatsReport.prototype.forEach = function(callback) {
    return this.values().forEach(callback);
};

[hthetiot]

Do you think master is stable enough to use in production as of now?

See https://github.com/cordova-rtc/cordova-plugin-iosrtc/issues/687

Once people confirm i will release.