search

samvera-labs / ramp

Interactive, IIIF powered audio/video media player React components library. Styleguidist Docs: https://samvera-labs.github.io/ramp/
https://ramp.avalonmediasystem.org/
28 stars 5 forks source link

Unusual Behavior For Captions in iOS Native Player #473

Closed joncameron closed 2 weeks ago

joncameron commented 3 months ago

Description

To Reproduce

Alternatively, to reproduce

Done Looks Like

joncameron commented 3 months ago

Changes made in the video.js player seem to be replicated in the native player but it's broken in this particular instance.

Dananji commented 1 month ago

In the second scenario in the description;

  • go to: https://avalon-dev.dlib.indiana.edu/media_objects/rr171x25z in iOS 17
  • play video
  • CC option is selected by default in the native player
  • select off in the subtitle menu (should turn off, but sometimes that doesn't work)
  • select English option in the subtitle menu
  • captions do not display in player
  • if I go back and try off again, and come back to english option, I have been able to get captions to come back on

From what I've experienced, when captions are turned off in the native player for HLS media, there's a set of specific steps to follow to enable them again. This was an issue we had with the previous MediaElement.js implementation as well.

Once captions are turned off in the native player, to turn them back on click CC option and then the captions options to be turned on. And this example English option.

Or this can be turned on the Video.js player after exiting the native player by just clicking on the English option in the captions menu.

elynema commented 3 weeks ago

From testing the first case listed above:

  1. Those French captions aren't turning on when first selected. I'm not sure if it is because they are listed below the CC option somehow? I wondered if it was because they were the second caption option listed, but that is not a problem with this record: https://avalon-dev.dlib.indiana.edu/media_objects/rr171x25z (notice that the second caption option does appear above the CC option in the native player).
  2. If the captions are disabled on the web view after being enabled in the native player, the native player gets confused in the Subtitles menu.
elynema commented 3 weeks ago

@Dananji Are you suggesting that we should have a known issue for turning off captions in the native player?

Dananji commented 3 weeks ago

Once captions are turned off, I think those specific steps needs to be followed in the native player to turn them back on for stream media because of the additional CC option.

So, yes I think we should document this as a known issue.

Dananji commented 3 weeks ago
  1. Those French captions aren't turning on when first selected. I'm not sure if it is because they are listed below the CC option somehow? I wondered if it was because they were the second caption option listed, but that is not a problem with this record: https://avalon-dev.dlib.indiana.edu/media_objects/rr171x25z (notice that the second caption option does appear above the CC option in the native player).

It seems captions cannot be simply turned on by selecting the caption option in the native iOS player's subtitles menu once they are turned off. And I don't think we can fix this, since we cannot access the native iOS player instance from the code.

For the second item mentioned in the comment, I'm only seeing one caption labeled English. However I tested another item with multiple captions (on this item the CC option was below the captions options) and it showed the same behavior as explained here. I think the order of the caption with respect to CC option does not affect the selections.

As you mentioned, when the video is played for the first time after turning captions off, the native player's subtitles menu randomly shows either CC or Auto options as selected. In the off chance when CC option is selected in the native player the French captions turned on the first time I clicked on it. But this was very random.

I think we need to add this as a known issue in both Ramp and in Avalon.

  1. If the captions are disabled on the web view after being enabled in the native player, the native player gets confused in the Subtitles menu.

I was able to reproduce this. Once captions are turned off in web view the native player still shows the French caption as selected. But captions text is not displayed on top of the video. After playing the video in the native player and exiting it; I checked the web view player, which still had captions turned off. And if I change the caption selection in the web view to English, the captions are displayed on playback in the native iOS player but the subtitles menu still shows French captions as selected. So the behavior for captions display on top of the video in the native player is consistent with what you see in the web view player, even though the selection in the subtitles menu in native player is incorrect.

I can take a look at this second issue, but I don't have a clear idea on how to fix it at the moment.

Dananji commented 3 weeks ago

I was not able to resolve the 2nd issue.

None of the fixes I explored did not update the caption selection in the iOS native player.

Caption selections made in the native player are reflected in the web view player (Video.js instance). And caption selections made in the Video.js player are reflected in the caption displayed on top of the video on playback in the native player but not in the selections in the subtitles menu.

@elynema and @joncameron, Since the user can change the caption selection via the web player and see the correct captions in the native iOS player, maybe we could mark this as a known issue for native iOS player?

elynema commented 3 weeks ago

Testing a few items today, I have found that sometimes when you turn the captions off in the web viewer, the native player auto-selects the CC option in the subtitles menu. When that happens, it seems that you can pick a subtitle and it will turn on in the native player the first time around. This kind of feels like just keep selecting things and eventually you will get what you want.

My attempt to summarize known issues to document:

@Dananji I ran into something that may be what you have already fixed today?

  1. I tested with third section on this record: https://avalon-dev.dlib.indiana.edu/media_objects/g158bh28p on iOS 17 on iPhone Safari
  2. I selected the non-default caption in the web player and then hit play
  3. Native player opened with captions playing, but the default captions were playing, not the captions I had selected in the web player.
Dananji commented 2 weeks ago

@elynema I was unable to reproduce this issue. I tested it multiple times for different iOS 17 versions in iPhone 15.

Steps I followed;

  1. Opened the item in Ramp demo site
  2. Selected the third section. I got the captions-example.srt as the default caption
  3. Selected lunchroom-manners.mp4.vtt caption from caption menu
  4. Click play
  5. Native iOS player opened and played the video with the selected captions

Are there any additional steps or any thing else that I'm missing?

However I noticed something odd for that particular section on desktop browsers. Sometimes both captions are shown as selected in the captions menu even though there's only one selected caption. I will work on fixing it.

joncameron commented 2 weeks ago

👍 I'll update the Known Issues wiki page regarding the native player behavior.

elynema commented 5 days ago

@joncameron I only see a link to https://github.com/samvera-labs/ramp/issues/472 on the known issues page for Avalon. Should there be more info provided?

@Dananji I don't see this behavior under the known issues for Ramp; do we need to add that to a Ramp deployment issue?