scientist-softserv / utk-hyku

Other
6 stars 0 forks source link

🦄 Spike: Include closed captions / transcripts in manifest #403

Closed jillpe closed 1 year ago

jillpe commented 1 year ago

New ticket as of 2023-06-22

Update is due to client feedback of not needing closed captions to display in the UV

Summary

UTK would like their videos to have the ability to turn on closed captions. There may be multiple languages to choose from, so they would like the ability to select which language to view also. Currently closed captions are not supported by the UV. The goal of this ticket is to determine the how we can get closed captions working.

Related

Accepted Criteria

Notes

A few resources to start from the IIIF Cookbook:

Old Ticket # Summary UTK would like their videos to have the ability to turn on closed captions. There may be multiple languages to choose from, so they would like the ability to select which language to view also. Currently closed captions are not supported by the UV. The goal of this ticket is to determine the how we can get closed captions working. # Related - [x] #386 - [ ] #393 # Accepted Criteria - [ ] Pathways to implement closed captions have been identified - [ ] A pathway has been selected, and follow up tickets have been created # Notes A few resources to start from the [IIIF Cookbook](https://iiif.io/api/cookbook/index.html): - https://iiif.io/api/cookbook/recipe/0219-using-caption-file/ - https://iiif.io/api/cookbook/recipe/0231-transcript-meta-recipe/ - https://iiif.io/api/cookbook/recipe/0017-transcription-av/ - https://iiif.io/api/cookbook/recipe/0074-multiple-language-captions/
ShanaLMoore commented 1 year ago

UV slack Q: https://universalviewer.slack.com/archives/C8WDXNPA7/p1686174552793529

ShanaLMoore commented 1 year ago

ref universal viewer issue for discussion: https://github.com/UniversalViewer/universalviewer/issues/747

And commit https://github.com/UniversalViewer/universalviewer/commit/5436bcddfeb345bca0ba9c19c4cf410e55706a93

Image

ShanaLMoore commented 1 year ago

from a convo with Lea Ann

Image

Would avalon replace the universal viewer?

ShanaLMoore commented 1 year ago

msg to Chris Colvard:

Image

His reply:

"I don't know about the state of VTT support in UV. In Avalon we currently use non-IIIF player built on mediaelement.js, but we're finally working on switching to our home grown IIIF player for AV content called Ramp (built on video.js). It supports VTT for subtitles, closed captioning, or transcripts. While it wouldn't be a consistent UX, it is relatively easy to dynamically choose the IIIF player to use in Hyrax based upon the work if you want to use ramp just for AV content. Myself of @dananji would be happy to answer any questions you might have or the #avalon channel is available for all AV questions even if they aren't specifically avalon related."

ShanaLMoore commented 1 year ago

https://developer.mozilla.org/en-US/docs/Web/Guide/Audio_and_video_delivery/Adding_captions_and_subtitles_to_HTML5_video

ShanaLMoore commented 1 year ago

See Mark's response in Google docs: https://docs.google.com/document/d/1GsoeK7gkBq3yziAVIK4Fijihd7jNo5nkc0Fnj8K1-B4/edit

For audio, are there transcriptions we should be aware of? If so, how do you want them to look?

There are timed text files, but the important thing is to just model them like you would on video. Since we’re using Universal Viewer for A/V, I don’t think we can expect “captions” to be “used”. What we want is for these to be in the IIIF manifest as an annotation that with a motivation of supplementing that targets the audio canvas.  That way, if we were to reuse the manifest in another application that supports IIIF audio annotations like Clover, they would show in that application:
https://samvera-labs.github.io/clover-iiif/?iiif-content=https://digital.lib.utk.edu/assemble/manifest/rfta/134
ShanaLMoore commented 1 year ago

Maybe consider replacing the UV with https://github.com/samvera-labs/clover-iiif

jillpe commented 1 year ago

confirmed client meeting 2023-06-21, we can do the same thing we are doing with the audio transcripts: we just need them to be available in the manifest, so when the UV is capable of doing closed captions it can pull from that. We do not need to get closed captions to appear in the UV as part of this work.

kirkkwang commented 1 year ago

closing this spike, implementation ticket: #442