Add VTT captions for AV content

[edsilv]

https://www.w3.org/WAI/WCAG21/quickref/?showtechniques=122%2C123%2C125%2C127%2C134%2C111#audio-only-and-video-only-prerecorded

https://www.mediaelementjs.com/

Annotate webvtt with a describing motivation.

[stephenmcconnachie]

I have lots of examples of video and associated webvtt files, if examples are required.

[edsilv]

@stephenmcconnachie Amazing! Here's one with webvtt annotated: https://iiif-commons.github.io/iiif-av-component/examples/data/iiif/lunchroom-manners.json Please add any examples to this issue - much appreciated.

[edsilv]

https://github.com/mediaelement/mediaelement/blob/master/docs/usage.md#setting-new-captions

[markpbaggett]

https://www.w3.org/WAI/WCAG21/quickref/?showtechniques=122%2C123%2C125%2C127%2C134%2C111#audio-only-and-video-only-prerecorded

https://www.mediaelementjs.com/

Annotate webvtt with a describing motivation.

Just adding to this as this is definitely something I'd like to see in UniversalViewer. It looks like the IIIF AV community has come to the decision that the motivation should be supplementing rather than describing as in the original post. See this issue. If helpful, here is a manifest that uses the supplementing motivation with a webVTT.

[stephenwf]

The current implementation doesn't need to know the motivation, just that its a WebVTT format: https://github.com/stephenwf/universalviewer/commit/5436bcddfeb345bca0ba9c19c4cf410e55706a93 from the content type. Is there a case where WebVTT would appear in the annotation body without that motivation and for it to be processed differently?

[DanOlson]

What is the status of this? Are captions supported now? I see the accessibility statement says they are not, but the merged commit 5436bcdd is newer and seems to add support. Please let me know, thanks!

[stephenwf]

It's implemented for AV Manifests without ranges, but the AV Component does not yet support them. So it's 50/50

[ShanaLMoore]

It's implemented for AV Manifests without ranges, but the AV Component does not yet support them. So it's 50/50

@stephenwf Has anyone identified the work needed for the AV Component to support this? If so, could you please link me? It's possible that my team may be able to contribute to these efforts.

[stephenwf]

@ShanaLMoore The code for the AV Component is here: https://github.com/IIIF-Commons/iiif-av-component, and the relevant area of the code where the <video/> tag is created can be found here: https://github.com/IIIF-Commons/iiif-av-component/blob/master/src/elements/media-element.ts

The AV component uses only a video tag, instead of https://www.mediaelementjs.com/ which has an option for supporting VTT captions. I'm not sure what would be required to either change the AV component to use media element, or how to support it without.

[LlGC-szw]

All issues will be triaged for further investigation or closure by the 28 September 2023. If your issue is still relevant and would like for it be investigated further please comment by 14 September 2023.

[DanOlson]

@stephenwf @edsilv I have a couple PRs which I believe together provide full coverage for captions on manifests with or without ranges. Please let me know if there's anything else I can do to help.

[demiankatz]

@DanOlson, @edsilv isn't going to be available to work on the project for a while, so the rest of the community is currently working on developing and documenting some formal processes to get things reviewed and merged at a more predictable pace. We're running slower than normal while we figure this all out, but progress is being made! There's going to be a Community Call today at 11am Eastern that you're welcome to join if you're interested (details on Slack -- let me know if you need more specific direction).

We're trying to use the Community Board to prioritize reviews; your #1047 is already in the queue there. If there's another that should be added, just let me know and we can get it into the pipeline.

[DanOlson]

Ah, thanks for the info, @demiankatz! I can appreciate the situation you're in. I also have https://github.com/IIIF-Commons/iiif-av-component/pull/95, which is related, but specifically addresses manifests with ranges, as they're presented using a different component.

I am not able to attend the call today, but I'll check Slack for any call related information.

Thanks!

[DanOlson]

@demiankatz I seem to have lost access to the UV Slack org, but I requested access again last week. Is there anything else I can do to revive my access?

Also, is it possible to add https://github.com/IIIF-Commons/iiif-av-component/pull/95 to the Community Board? This rounds out caption support and between that and #1047, I think this issue could probably be resolved. Thanks!

[demiankatz]

@DanOlson, I see your request, but Slack didn't re-add you because you have an existing account. (I tried to do it manually, but it just says that you're already there). Is there not a "forgot password" option? I would think that should work if your email address is still valid. But if for some reason that isn't working, I can investigate whether there are any other possible solutions -- maybe I could delete your account before re-adding you, for example.

Regarding IIIF-Commons, I'm not sure who controls those repositories -- I think @jamesmisson was investigating that situation since he is also trying to get some fixes and improvements merged into repos there. I don't think I have the necessary permissions to help with the commons, and I wasn't sure if those PRs were in scope for the UV board (since it's a whole different organization, I'm not sure if it will work... but I'll at least try after I submit this comment).

[demiankatz]

Okay, I was able to add the issue to the board! I'm not sure if this will be the right workflow going forward, but at least this will remind us to figure it out. :-)