search

open-webrtc-toolkit / owt-client-native

Open WebRTC Toolkit client SDK for native Windows/Linux/iOS applications.
https://01.org/open-webrtc-toolkit
Apache License 2.0
384 stars 180 forks source link

OWT Fails to Stream Full HD (1920x1080) Video on newer iPhones than iPhone 11 #724

Open rouzbeh-abadi opened 11 months ago

rouzbeh-abadi commented 11 months ago

We have been using the OWT iOS SDK to build a real-time video streaming application. We recently encountered an issue where full HD video streaming (1920x1080) works fine on iPhone 11 but fails to work on newer iPhones, including iPhone 13. The video freezes as soon as it reaches the resolution of 1920x1080. However, lower resolutions such as 1280x720 work without any issues.

Upon investigation, we have narrowed down the issue to the integration with the OWT SDK. We believe that there might be a problem with the OWT SDK's handling of higher-resolution frames on newer iPhones.

Our implementation involves creating pixel buffers and frames from the captured frames, which are then passed to the OWT SDK using the videoSource.capturer method.

We have confirmed that the frame size is correct for both iPhone 11 and newer iPhones (8294400 bytes). However, the freezing issue only occurs on newer iPhones, indicating a potential problem in the OWT SDK's handling of these frames.

We would greatly appreciate any insights or guidance on how to address this issue. Thank you.

Expected Behavior:

Steps to Reproduce:

  1. Run the application on an iPhone 11 and initiate full HD video streaming (1920x1080).
  2. Observe that the video stream functions correctly without any freezing or issues.
  3. Repeat the same steps on an iPhone 13 or another newer iPhone.
  4. Notice that the video freezes as soon as it reaches the resolution of 1920x1080, rendering the streaming unusable.

Environment:

OWT iOS SDK version: 5.0 iOS version: 16.5.1 Device models affected: iPhone 13 and other newer iPhones

Please let us know if any additional information is required to investigate and resolve this issue. Thank you for your assistance.

jianjunz commented 11 months ago

Thanks for reporting this issue. But we don't have iPhone 13 or newer iPhones for testing right now. The default renderer is implemented in libwebrtc, you can try to use the latest main branch as we upgraded libwebrtc to m108 branch, which is newer than the branch used by 5.0. You can also create your own renderer by implementing RTCVideoView interface.

rouzbeh-abadi commented 11 months ago

Thank you for your previous response. I would like to clarify our issue further. We are using camera feed to display the video stream on our iOS devices, and there are no issues on the iOS side. The video plays smoothly at all resolutions on our iOS devices, including those above 1280x720.

However, when we transmit video frames above 1280x720 resolution, the receiving end of the stream starts to freeze. Considering that we're directly using the camera feed, implementing a custom RTCVideoView does not seem to be a relevant solution for us.

We suspect the issue may lie within the OWT SDK's handling or processing of high-resolution video frames on the receiving end. Could you please provide any insights, known issues, or potential solutions regarding this?

Thank you