search

cordova-rtc / cordova-plugin-iosrtc

Cordova iOS plugin exposing the WebRTC W3C API
MIT License
688 stars 340 forks source link

Pretty bad H264 performance after upgrading to iOS 15.4 #744

Open derMani opened 2 years ago

derMani commented 2 years ago

Description

Hi everyone.

after upgrading our phones to iOS 15.4, we are experiencing a pretty bad H264 performance. Versions before iOS 15.4 with the same source code are not affected by this provlem

I can provide a simple peer 2 peer example if needed

Console output looks normal.

Versions affected

Steps to reproduce

Establish an iPhone to Chrome video connection via H264 and compare it with VP8. A lot of jitter and bad FPS situations will occur, if you observe chrome://webrtc-internals

Both H264 profiles in the SDP with cordova iosrtc are affected

Expected results

Fluid video image, like in previous versions. Maybe an upgrade of the webrtc lib could help?

Actual results

Stuttering video quality with H.264

hthetiot commented 2 years ago

Versions before iOS 15.4 with the same source code are not affected by this provlem

I don't think compilation of WebRTCLib will help here, and BTW, you don't need that plugin to use WebRTC using iOS 15.4.

Try without the plugin (comment register global) to see if Native iOS WebRTC also have the same performance issue.

derMani commented 2 years ago

Update: Without using the globals of cordova-iosrtc: H264 works just fine with "native" WebRTC of Webkit.

Pros:

Cons:

The plugin had a good run, but I think most users of this plugin can now move on to the "native" WebRTC of Webkit

Anyway: The libwebrtc bug remains for H264 performance. This also happens in non cordova based projects when using libwebrtc

hthetiot commented 2 years ago

Thank you @derMani

The plugin had a good run, but I think most users of this plugin can now move on to the "native" WebRTC of Webkit

I agree, since iOS 15+, you don't really need the plugin if you don't have to put speaker on or support old iOS versions. Still you will have to deal with know ios safari webrtc issues that the plugin handle to manage (auto play, speaker, background).

hthetiot commented 2 years ago

In any case I will leave this issue open to.see if updating to latest libwebrtc fix thr issues.

hthetiot commented 2 years ago

@derMani see webrtc-100 PR that update webrtc to v100 that pass the CI (include full build test https://github.com/cordova-rtc/cordova-plugin-iosrtc/blob/master/.github/workflows/main.yml#L59)

shrhoads commented 2 years ago

I think this project may still be useful if the H264 encoding issue can be fixed. It would be nice to be able to provide higher quality encoding than is available in the safari webrtc support.

hthetiot commented 2 years ago

@shrhoads

I think this project may still be useful if the H264 encoding issue can be fixed. It would be nice to be able to provide higher quality encoding than is available in the safari webrtc support.

Then, can you then try #748 to see if iosrtc rendering from h264 is better?

hthetiot commented 2 years ago

Ping @derMani @shrhoads

https://github.com/cordova-rtc/cordova-plugin-iosrtc/issues/744#issuecomment-1129085749

dl4mmers commented 1 year ago

@shrhoads

I think this project may still be useful if the H264 encoding issue can be fixed. It would be nice to be able to provide higher quality encoding than is available in the safari webrtc support.

Then, can you then try #748 to see if iosrtc rendering from h264 is better?

Hey @hthetiot,

I've tested PR #748 on iPhone 14 Pro (iOS 16.3.1) and it fixes the H264 performance and fps issues.

iosrtc_h264_fps_m100_vs_6 0 21

derMani commented 1 year ago

Hi together, sorry für the late reply. I can confirm, that this works now without stuttering (also on iOS16) 👍

hthetiot commented 1 year ago

Hi together, sorry für the late reply. I can confirm, that this works now without stuttering (also on iOS16) 👍

@derMani using #748 also ?

In any case I'm going to release it as soon as possible.

derMani commented 1 year ago

Yep, #748 works fine 👍

hthetiot commented 1 year ago

Yep, #748 works fine 👍

Thx this help a lot to validate build.

dl4mmers commented 1 year ago

Hey @hthetiot, I continued testing PR #748 with my application and I noticed that, while H264 works fine, some WebRTC API's do not work like with iosrtc@6.0.21. I then tested iosrtc@8.0.4 and it's the same case:

After initiating a call and sending audio + video from iPhone 14 Pro to chrome browser

are always returning empty arrays in PR #748 and iosrtc@8.0.4, while with version iosrtc@6.0.21 there are correct RTCRtpSender objects inside. In chrome there are also correct values. So, when upgrading to latest iosrtc, I would have to use deprecated functions RTCPeerConnection.getLocalStreams() and RTCPeerConnection.getRemoteStreams().

Is this a bug?

hthetiot commented 1 year ago

Thank you @dl4mmers that really helpful. I'm a bit surprised of this issue. Let me investigate.

@RSATom would you mind check the comment above ?

RSATom commented 1 year ago

@hthetiot we are using WebRTC library from Chrome v102 pretty long time without any significant issues. But I have to check if we are using getSenders() | getReceivers() | getTransceivers()... I'll try to find time for that soon.

hthetiot commented 1 year ago

@RSATom can you do a PR for WebRTC 102? We have PR for 100 already here #748

hthetiot commented 1 year ago

@dl4mmers do you know if you use webrtc-adapter that may be the cause of the issue, I have seen thr shim F* iosrtc sham in the past ?

RSATom commented 1 year ago

@hthetiot I think I did PRs for all my changes to iosrtc (and all of them was accepted if I remember correctly). What about WebRTC lib - we are using patched one https://github.com/WebRTSP/WebRTC/commits/m102-patched https://github.com/WebRTSP/libWebRTC/releases And I don't know if it's good idea to use that patched library version in iosrtc.

hthetiot commented 1 year ago

@RSATom OK I will build or may be move to the jisti one use by react-native-webrtc and made original maintainers of iosrtc.

dl4mmers commented 1 year ago

@hthetiot Yeah I'm using webrtc-adapter, but I've looked through the swift code of RTCPeerConnection (https://github.com/cordova-rtc/cordova-plugin-iosrtc/blob/master/src/PluginRTCPeerConnection.swift#L324) and saw the check IsUnifiedPlan() and then noticed that I'm still using plan-b on iOS. I set it to unified-plan. Now with sdpSemantics: 'unified-plan' I get correct objects via RTCPeerConnection.getSenders(), RTCPeerConnection.getReceivers() and RTCPeerConnection.getTransceivers().

hthetiot commented 1 year ago

I will set unified-plan by default in next release I think it is time. Will avoid wasting people time.

dl4mmers commented 1 year ago

In any case I'm going to release it as soon as possible.

Hey @hthetiot, any news on releasing 10.x with WebRTC M100 or M102?

hthetiot commented 1 year ago

@dl4mmers yes, just need testing on that PR https://github.com/cordova-rtc/cordova-plugin-iosrtc/pull/748 see testing instructions in description. I will rebase master this wk and re ping here. I don't want to release without community feedback to risky.