Call fail using latest sip.js in ionic app

[MarwaAbuEssa]

YOU MUST read first!

Please use Community Forum for general technical discussions and questions.

• [x] I have used Google with the error message or bug in association with the library and Cordova words to make sure the issue I'm reporting is only related to iOSRTC.
• [x] I have provided steps to reproduce (e.g. using sample app code https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample or updated extra/renderer-and-libwebrtc-tests.js file).
• [x] I have provided versions of third party library name, ios, Xcode and plugin version and adapter.js version if used.

Versions affected

• Capacitor 3 iOS Platform version (15):
• Plugin version (latest):
• iOS version (e.g 15.0.1):
• Xcode version (e.g13):
• WebRTC Framework version (e.g. SIPJS 20):

Description

HI ,

our VOIP app (ionic 5 and capacitor) using sipjs v20 and there is many issues using webrtc with callkit (specially with audio stream locked by ios callkit)

I thought to use your plugin to produce my ios app version and adopt callkit with it .

I installed the plugin successfully , did global registration , enumerate devices and get media stream. my issues are : 1- since upgrade to ios 15 here is an issue with audio tracks (although i can get the input device) and i do think this is related to upgrade to v15 2- how can integrate your plugin with sipjs , I means should should i do the invite from spjs but get the stream and session description from your plugin ?? I need some clear examples about appropriate way to use the plugin with sipjs and also if the plugin is working with ios callkit

Thanks in advance , your help is really appreciated as I have been long time investigating with ios and webrtc

Marwa

[hthetiot]

Note: If the checkboxes above are not checked (which you do after the issue is posted), the issue will be closed, removing this checkbox will result in automatic closed issue.

https://github.com/cordova-rtc/cordova-plugin-iosrtc/blob/master/.github/ISSUE_TEMPLATE.md

First Update your issue with proper information including checkbox.

Secondly, if you actually had looked at the sample in th issue posting requirements, you would have seen a JSSip sample is included https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample/blob/master/www/js/index-jssip.js

how can integrate your plugin with sipjs Read the README documentation properly, once you called the registeeGlobal API there is nothing special todo.

[MarwaAbuEssa]

Hi,

Thanks a lot for your reply .my apologizes for missed data . actually I tried to run the sample on iphone but I couldn't , that's why I asked for a sample code.

Could I ask if you used your Plugin works with iOS callkit (with using SIPJS or JSSIP) ? I means you can make a call/receive from VOIP app and display it at iOS callkit and earpiece and mic is working fine?

Thanks a lot Marwa

[hthetiot]

Thank you for updating the issue.

I'm aware of people using iosRTC with callkit without issue. It may depend the version of the callkit cordova plugin.

What was your issue with the sample ? You just need to change the bundle name and let certificate generate in Xcode.

If you want pay support you can always contact support @ sylaps.com, i can only provide support here if i can fully reproduce the issue and spent a minimum of time. I use this plugin with Sip.js (not JSSip) but tht sample is made for both and worked last time I tested few weeks ago.

[MarwaAbuEssa]

Hi ,

Thanks a lot for your reply .I tried callkit cordova plugin and its working with incoming call the audio is working perfect with earpiece and mic but if the user lost focus on app (for example opened another app while running my VOIP app) the callkit still working with earpiece but it loses mic , the other side of call can't hear any thing . for the outgoing call , if I started the call from callkit before sipjs invite it gives NotReadableError: The I/O read operation failed if I started the call from sipjs invite then from callkit , the mic stream is not accessable from callkit and the same issue of incoming issue happens .

I also prefer SIPJS but I'm really sorry I'm still a little confused about using SIPJS , I will register user agent using sipjs mainly then I'm going to send iosrtc the rtconnection ? or the reverse I should init rtconneection with iosrtc then use it with sipjs ? also when doing outgoing call , should I use normal sipjs invite or use a special invite from iosrtc plugin ? really sorry for the confusion and I hope you got my point

Thanks in advance, Marwa

[hthetiot]

If you call cordova.plugins.iosrtc.registerGolbal you should be able to use JSSIP sam's has you do in browser.

For audio issues while all lose focus, you want to add the cordova.plugins.backgroundMode see https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample/blob/master/www/js/common.js#L654

Try that and let me know.

[MarwaAbuEssa]

Thanks a lot , I will try the background mode and send you . Just for confirmation , I will keep all my code for sipjs for registration , invite , refer ..... etc and only do the registerGolbal when app is ready ?

Thanks

[hthetiot]

Just for confirmation , I will keep all my code for sipjs for registration , invite , refer ..... etc and only do the registerGolbal when app is ready ?

yes all websocket communication stay the same, sipjs will make WebRTC api calls that we expose via registerGlobals when onDeviceReady, if you don't use registerGlobals, it will only works on iOS 14.3+ using native WebRTC.

Best is try sample and update index.html to use index-jssip.js to be used instead index-local.js.

[hthetiot]

Closing for now. Comment for re-open if you have other issue like crash.

[MarwaAbuEssa]

HI , Thanks a lot for your support , I added the Plugin and use registerGlobal but when I do an outgoing call (ios15) , it dropps immediately with the following error

at the invite of sipjs I'm using the following constraint

let inviteOptions:SessionInviteOptions = { sessionDescriptionHandlerOptions: { constraints: { audio: true, video: false} } };

await inviter.invite(inviteOptions);

the same happens when answering incoming call

the app is not prompt for mic permission although it prompt for camera permission .I have a way to give the app permission to access the microphone but not prompt for using it . any help ? Marwa

[MarwaAbuEssa]

I uninstalled the app and installed it again from xcode so after global register ,I called requestPermission for the plugin as following and the call back value is true but the app on ios is not getting permission for mic

cordova.plugins.iosrtc.requestPermission(true,true,function(requestpermission){ console.log(requestpermission , ' requestpermission requestpermission'); });

any reason for mic issue or missing configuration from my side?

[MarwaAbuEssa]

I can list all media devices successfully , getusermedia without any errors but the app is not prompt for mic permissions with the iosrtc plugin?

[hthetiot]

I can list all media devices successfully , getusermedia without any errors but the app is not prompt for mic permissions with the iosrtc plugin?

It will ask once, not multiple time, if you see yourself in video you have no issue.

[hthetiot]

HI , Thanks a lot for your support , I added the Plugin and use registerGlobal but when I do an outgoing call (ios15) , it dropps immediately with the following error

at the invite of sipjs I'm using the following constraint

let inviteOptions:SessionInviteOptions = { sessionDescriptionHandlerOptions: { constraints: { audio: true, video: false} } };

await inviter.invite(inviteOptions);

the same happens when answering incoming call

the app is not prompt for mic permission although it prompt for camera permission .I have a way to give the app permission to access the microphone but not prompt for using it . any help ? Marwa

This error has nothing to do with GetUserMedia, you SDP is rejected most likely because of a configuration issue in you SIP server.

[hthetiot]

I cannot help you more sorry this getOffer rejected come from your usage of the library or configuration of your SIP server

[MarwaAbuEssa]

Thanks a lot appreciate your help

[RaddishIoW]

@MarwaAbuEssa I am having exactly the same problem you are in the screenshot above... did you find out what was causing it, or have any pointers for a solution?

[MarwaAbuEssa]

Hi @RaddishIoW

I figured out the cause of the issue that sipjs uses MediaStreamTrackEvent datatype to dispatch event on addtrack/removetrack function which is different type of event returned from iosrtc . I solved it by doing 2 line modification in sip

1- go to file node_modules\sip.js\lib\platform\web\session-description-handler.js and replace the following

static dispatchAddTrackEvent(stream, track) { stream.dispatchEvent(new MediaStreamTrackEvent("addtrack", { track })); } to static dispatchAddTrackEvent(stream, track) { stream.dispatchEvent(new Event("addtrack", { track })); } and

static dispatchRemoveTrackEvent(stream, track) { stream.dispatchEvent(new MediaStreamTrackEvent("removetrack", { track })); } to static dispatchRemoveTrackEvent(stream, track) { stream.dispatchEvent(new Event("removetrack", { track })); }

I know that this solution is improper but I had been so busy lately to change it. hopefully, this can help you. Thanks

[hthetiot]

Thank you @MarwaAbuEssa,

May be we can mitigate that on cordova side somehow by updating iosrtc.

PR is welcome 🙏 alternatively I might look at it next time I allocate time to iosrtc.

I will leave it open for now so it can be found more easily. I will hide non relevant comments and update titles to help issue context.

[RaddishIoW]

@MarwaAbuEssa Thanks so much for that!

I know that sip.js has a mechanism for creating your own SessionDescriptionHandlers, so I could probably create a SessionDescriptionHandler specifically for Cordova with this change in it, using the standard web one as a starting point.

[hthetiot]

@RaddishIoW thx

Would be great if we can update the jssip/sip.js iosrtc example app (https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample) with this custom session Description handler thing.

See jssip example that you may duplicate for sip.js only that require to update www/index.html to use index-jssip.js instead of index-local.js

Index-jssip file for reference: https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample/blob/master/www/js/index-jssip.js

Like that I can test for regression on each release. In the past this sample worked for both depending what jsSipUrl value was (sip.js or jssip cdn url) because sip.js is a fork of the much more stable jssip, it had to broke when they updated, it was working for sip.js version 0.15.6 at the time.

Note: Edited for typos and better context and wording.

[RaddishIoW]

Doing some digging on this... The MediaStreamTrackEvent event datatype is defined in the WebRTC APIs. https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamTrackEvent

I'm probably missing something, but shouldn't iosrtc support this event datatype?

It would be possible to define a SessionDescriptionHandler with the changes above, but unfortunately the methods are marked private, so subclassing the platform/web/session-description-handler class won't work, as we can't override the methods... so we would have to duplicate the whole class' code in order to change those dispatch methods - which isn't ideal.

Thoughts?

[hthetiot]

Doing some digging on this... The MediaStreamTrackEvent event datatype is defined in the WebRTC APIs. https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamTrackEvent

I'm probably missing something, but shouldn't iosrtc support this event datatype?

Iosrtc is a SHIM, we can try define this event "type" or even try to use it if webrtc is available as it currently use native Event I'm not sure why it does not work. As any event can be dispatch like that in browser (you dont need MouseEvent to trigger click for example https://developer.mozilla.org/en-US/docs/Web/Events/Creating_and_triggering_events).

They may be checking somewhere the type some how using constructor.name. None other client webrtc library does that to my knowledge, might be more easy to fix on their side.

On iosrtc side change would need to be made there:

• https://github.com/cordova-rtc/cordova-plugin-iosrtc/blob/a502fccd274d4e793e3195b24bd990d9eb53af40/js/MediaStream.js#L448

And we would need something like (not tested) to declare this CustomEvent.

const MediaStreamTrackEvent = window.MediaStreamTrackEvent || function(type, eventInitDict) {
    this.type = type.toString();
    this.track = eventInitDict.track;
    return Event.prototype.constructor.call(this, type, eventInitDict);
}

Object.defineProperties(
    MediaStreamTrackEvent,
    Object.getOwnPropertyDescriptors(Event.prototype)
);

MediaStreamTrackEvent.prototype.constructor = MediaStreamTrackEvent;

I'm curious doh, can we confirm you load iosrtc registerGlobal BEFORE sip.js load, because that prevent bad reference to be made in sip.js library (see index-jssip.js referenced earlier) this as solve such problems in the past for twilio library.

[hthetiot]

@RaddishIoW

private, so subclassing the platform/web/session-description-handler class won't work

Private does not really exist in JavaScript ES6, subclassing via pure JS will work using Object.defineProperties (typescript does not exist its a synax sugar to build ES6 code, that why iosrtc is pure ES6 to avoid having debugging compiled ES6 in the end, class are prototype constructor in fact in ES6 and TS).

Note: before you ask, I have nothing again TypeScript but this projet will not use it to keep code as light and close to the native JavaScript to avoid side effects.

[MarwaAbuEssa]

Thank you @MarwaAbuEssa,

May be we can mitigate that on cordova side somehow by updating iosrtc.

PR is welcome 🙏 alternatively I might look at it next time I allocate time to iosrtc.

I will leave it open for now so it can be found more easily. I will hide non relevant comments and update titles to help issue context.

Thank @hthetiot for reopening it . yes update iosrtc Is more suitable .I will try to do my best

[MarwaAbuEssa]

Hi @RaddishIoW

sipjs allows to create a sessiondescriptionhandler but with your custom mediastreamfactory .so mysolution was to use iosrtc mediastream and provide it to sipjs . you can find override how local media section at the following link

https://github.com/onsip/SIP.js/blob/main/docs/session-description-handler.md

of-course you need to define iosrtc globally in your project first and do the mediastreamevent modification too.

Regards

[hthetiot]

exposed MediaStreamTrackEvent via via b5a3bb0ab6fb4e32d54eb7a80bb3259192814de1

will be release on 8.0.4

[hthetiot]

Released into cordova-plugin-iosrtc@8.0.4 , can you test to confirm your fix please, thx.

https://github.com/cordova-rtc/cordova-plugin-iosrtc/blob/master/CHANGELOG.md#version-804

@MarwaAbuEssa @RaddishIoW

[hthetiot]

Ping @MarwaAbuEssa @RaddishIoW pls test 8.0.4