search

twilio / twilio-video.js

Twilio’s Programmable Video JavaScript SDK
https://www.twilio.com/docs/video/javascript
Other
571 stars 217 forks source link

iOS 12.2 Safari on Video Rotation (rotating right) Remote Video Freezes with Group Rooms #637

Closed browniefed closed 4 years ago

browniefed commented 5 years ago

Bug Information Rotating the phone to the left lets rotation happen and video doesn't freeze on remote side.

Testing Peer-To-Peer rotating to the right/left works and video doesn't freeze for remote participant. Testing Group Rooms rotation to right will cause remote participants video to freeze.

When rotating to Portrait or rotating to the left and entering landscape mode video for remote participant returns to normal.

This was with H264, and saw the same issue when testing VP8.

This only occurs on phones with iOS 12.2. I tested on iPhone S, 6s, 7 all running iOS 12 or below and things worked correctly.

Once updating the iPhone 7 to iOS 12.2 I ran into this bug.

The video on the mobile device will continue in landscape mode as normal, only the remote participant (tested on Chrome/Firefox) will have video frozen.

Other Notes We listen for dimensionsChanged and log that to console on Remote Participant side. I am not seeing this log coming in when the video enters into a frozen state when the phone is turned to the right landscape mode.

Turning to the left I see the dimensionsChanged event coming in.

Expected behavior: I expect the remove participant video to not freeze when rotating phone to the right and entering in landscape

Actual behavior: Remote participant video will freeze.

Software versions:

browniefed commented 5 years ago

Looking into this further. Trying to create a simple repro with video-quick-start but having a little trouble doing so.

I will continue to look into any logging/stack traces I can obtain or anything we may be doing incorrectly.

browniefed commented 5 years ago

@manjeshbhargav I've added a repro but it's for rotating the phone to the left here.

https://github.com/browniefed/grouproomtest

Simply commenting out type: 'group' fixes this. I don't know why it's a different direction in this particular case but rotating only one direction works. The remote participant (on PC) will freeze.

manjeshbhargav commented 5 years ago

Hi @browniefed ,

Thanks for writing in with this issue. I will investigate further and get back to you soon.

Thanks,

Manjesh Malavalli JSDK Team

browniefed commented 5 years ago

I realized that the reason it is left in the demo is because it's front facing camera. On our app it's the environment facing camera so turning right causes the freeze.

manjeshbhargav commented 5 years ago

Hi @browniefed ,

I've asked our QA team to verify the behavior you have described. I'll update you with our findings when they get back to me.

Thanks,

Manjesh Malavalli JSDK Team

manjeshbhargav commented 5 years ago

Hi @browniefed ,

Our QA team was able to reproduce this behavior. We have an internal JIRA ticket, JSDK-2358, tracking this bug. I will keep you posted on further developments.

Thanks,

Manjesh Malavalli JSDK Team

ceaglest commented 5 years ago

Hi @browniefed,

After some internal investigation, there seems to be an issue with stripping of CVO (rotation) tags in Safari on iOS 12.2. This capability is required in a Group Room, where there is no guarantee that all connected Participants support CVO tags. When CVO tags are stripped, the frames must be physically rotated to compensate.

In a Peer-to-Peer Room, as long as both Participants support CVO tags, there is no need to strip them and the bug does not occur.

I am investigating the WebKit source code (which uses large portions of Chromium WebRTC), to determine where the issue is occurring.

Best, Chris

ceaglest commented 4 years ago

Hi,

There is a WebKit bug (#201315) that is tracking the issue with CVO tag removal. It is claimed to be resolved as of iOS 13.0 and above. We will re-test JSDK-2358 to make sure.

Thanks, Chris