IOS doesn't work with the Descriptions on

[AlanZenyth]

Hi @terrill!

We are using Able Player, and so far, we could implement some fixes and workarounds for some accessibility issues we found, but there are some related to some speechsynthesis events from the Web Speech API we couldn't make work on IOS.

The example we are working on is https://ableplayer.github.io/ableplayer/demos/video3.html.

## We are struggling with the following issues:

### Issue 1 On IOS, the description only works once. Once the speech is queued, it seems we cannot cancel it, even turning off the device. The result of this is that the audio description doesn't work on IOS devices at all. After we tried to use the descriptions once, the main video (Now with descriptions OFF) stops at the point where the description should start. And the video won't play again.

The solution we found was to remove the Descriptions Button until we fix this issue.

### Issue 2 For the desktop version, in order to comply with 2.2.2: Pause, Stop, Hide, we are using the speechsynthesis.pause() and speechsynthesis.resume(), with a button, so the user can pause and resume the speech at any time.

We tried the exact solution that worked on desktop to implement some workarounds for Issue 1, but it didn't work.

We tried to "force" speechsynthesis.pause(), speechsynthesis.resume(), and speechsynthesis.cancel() with extra buttons implemented for testing, as we did for the desktop, but it didn't work for IOS.

But all of those events should work on IOS 7 and above, as you can see here: https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API#api.speechsynthesis

Is there any progress on this issue related to speech synthesis? Is there any live example of the Able Player VTT Descriptions working on IOS devices? Even with some limitations?

[terrill]

Hi @aspilhere,

Be sure you're using the latest version from the develop branch. There have been a lot of improvements made related to speech synthesis. Among others, audio description is now in sync with the media, so if users pause, fast forward, restart, etc., while audio description is being voiced, the description will stop as expected.

At least, that seems to be working reliably on desktop browsers. I haven't done much testing yet on mobile devices, but both are high on my todo list this week. So if you want to wait until I think it's working on iOS and Android, I can report back, at which point you can do your own testing and confirm.

[terrill]

@aspilhere this is much more reliable now in dbd95fa6d7e70b9909d5b57e44f3f5359ebeb9f0

iOS doesn't allow speech synthesis unless the user specifically requests it by clicking or tapping something. This prevents us from initializing speech when the player is built, so we have to do it on-the-fly after the user takes some action. I've carefully gone through the logic in Able Player and done a lot of testing. Speech output is now initialized the first time a user taps play, or the preferences button, or anywhere else on the page outside of Able Player. It builds an array of available voices in the current caption or player language, selects a voice from the array, and reliably speaks description content at the appropriate times.

I'm confident this is working reliably now so I'm closing this issue, but if you observe more problems please share the details here and we can reopen the issue and investigate.

[AlanZenyth]

Hi @terrill!

I'm the developer working on the Disability Justice Project player.

Thanks so much for the update! We tested it on IOS 11 and 12, and it worked! But we have a similar issue from before your last update for an IOS 10, the video plays and when it gets to the point it should trigger the description, the video stops, and we have to refresh the page to be able to play it again.

We tried to interact with the player before playing the video (open preferences, choosing a different language) but it still didn't work.

We are using all the latest files, and we also tested on the live example (https://ableplayer.github.io/ableplayer/demos/desc1.html).

Thanks so far!

Cheers

Alan

[terrill]

@AlanZenyth what is the minimum version of iOS that you're targeting? iOS is now up to 16.x, so I think it's reasonable to expect users to be more current than iOS 10. That said, Able Player should fall back gracefully if a device doesn't reliably support speech synthesis. How are you testing with iOS 10? Do you have an older iPhone or iPad, or are you using a simulator?

[AlanZenyth]

Hi @terrill! Thanks for replying! We tested on 2 newer devices (https://i.imgur.com/VAvQ1Je.png, https://i.imgur.com/nBnRhIY.png), and they worked fine. But when tested on an iPhone 10 with iOS 15, it didn't work and didn't fall back. We haven't tested it on the simulator. The problem seems to be related to when the description is activated. When we start the video with the description on, it seems to work even on older devices (I had to refresh the page a couple of times for older devices), but when we activate the Description while the video is playing, the Description doesn't work, and the video keeps paused. And when we activate the Description while the video is playing, sometimes the video pauses and also keeps paused, when a Description should be read by the browser. Also, when we activate the Description in a moment where no Description should be in place, the video pauses on the next Description, the same way as described before.

Thanks so far for your attention on this Terril, let me know if you can't reproduce those behaviors. I just tested it on https://ableplayer.github.io/ableplayer/demos/desc1.html.

KR

Alan

[terrill]

@AlanZenyth thanks for those details. That will help with troubleshooting...

[AlanZenyth]

Hey @terrill! I hope you're doing great! Just an update, I made a workaround for this issue; I made the player restart the video when the Description is triggered, I tested it on an old iPhone, and it worked so far.

Cheers

Alan

[AlanZenyth]

Hey @terrill! Just checking in. I hope everything is going well. We have found a workaround that might help you. If we open the settings once the transcripts work, it triggers the APIs.

I'm sure you're swamped, but if you could find a moment to reply, that would be awesome!

KR Alan