Enhanced Accessible Media Workflow

[JeffersonBledsoe]

PLIP (Plone Improvement Proposal)

Responsible Persons

Proposer: Jefferson Bledsoe

Seconder:

Abstract

This PLIP aims to deliver a new media uploading and viewing experience to improve the out-the-box accessibility of video when compared to the existing video block, whilst also providing the same level of functionality for audio which is not catered to by Volto currently.

Motivation

The current video block has a number of missing accessibility features such as the ability to add transcriptions, captions and alternative versions of the video for things like Audio Description. The block is based on Semantic UI React's Embed component, which also has its own accessibility issues. We do not have an 'audio' block in Volto and so the same accessibility issues that impact video also impact audio.

Assumptions

• We will focus on a React-only implementation for now. The chosen video player (discussed in the implementation section of this PLIP) does support other frameworks, however this will be additional work which will block the improvements this change could make to the out-the-box accessibility of Volto.
• Plone Classic will not receive this functionality in this initial implementation. Plone Classic’s primary solution to video and audio is wildcard.media, which provides a number of the features proposed in this PLIP, such as the ability to provide subtitles, upload media and write transcripts.

Proposal & Implementation

Which media player to use

We evaluated AblePlayer, plyr, MediaElement and video.js.

• AblePlayer is jQuery based and doesn't seem easy to integrate into the build due to it being a bit old. It also looks quite outdated. However, it had the best accessibility features along with video.js.
• plyr has a large number of features for video, however its Audio Description support isn't the greatest.
• MediaElement doesn't really have AudioDescription support either.
• video.js seems to have the best support for accessibility features after AblePlayer whilst still being modern and should be relatively into Volto, especially as it has React documentation. It also has the ability to play audio-only without many changes. video.js is a JavaScript enhancement for the native \<video> and \<audio> elements and so should still work if anything with the video player JavaScript goes wrong.

We are proposing to use video.js for this improvement.

Editing experience

• We will provide the ability for users to upload any number of video or audio files for a single media element.
  • This is needed to support Audio Description versions of the video to meet WCAG 2.2 SC 1.2.5 without the user having to manually figure out how to link the Audio Description version back to the original video.
  • Volto will work out whether the media block needs to be in video or audio mode based on the uploaded files.
  • We could also support using a URL instead of uploading a video, such as a Vimeo or YouTube link.
• We will provide the ability for users to upload WebVTT files to provide time-synchronised captions for a particular version of an uploaded media in order to meet WCAG 2.2 SC 1.2.2.
  • We could also support uploading SRT files as some caption-writing software exports these. Browsers do not support this file type universally, so we would need to convert the SRT to WebVTT. This conversion will only support basic cue files and won't have support for non-normative (but commonly used) WebVTT features like caption styling, chapters and comments.
• We will provide the ability for users to upload a text file containing the transcript for a particular version of an uploaded media to provide a text-only version of media in order to meet WCAG 2.2 SC 1.2.1.
  • This could be replaced with the ability to edit the transcript directly. Future versions upgrades may be able to support both but only one method will be used for the initial version to simplify implementation.
  • I am unsure how to best provide i18n support for these transcripts currently.
• The video block will be updated to support picking a media content from the object browser, making it easy to edit the selected media content object.
  • The user will need to decide whether they only want to display a link to the content object on the page (where we the viewer can see all of the various versions and transcripts) or if they want to display a particular version of the media inline too.

Viewing experience

• The media type can either be linked to directly by the video block, or it can display a version of the video directly on the page the video block is used.
• When viewing the content type, we would see all versions of the media on the page.
• We will render a \<details> element to display the transcripts. This could include the hidden=until-found attribute so the transcripts are able to be found by search crawlers and by the browser's find in page functionality.

Implementation

• Create a media content type in plone.volto.
  • Schema would include the title of the media and a list of the versions of media.
  • Schema for the versions would include:
  • The title of the version
  • A choice of the language of the version
  • An upload for the video or audio file
  • A transcript field and a caption field
  • The first version of the media will be the default
• Implement video.js into a new media block for rendering the video
  • Decide whether to render \<video> or \<audio> element from Volto based on the media versions.
  • Video tracks for video uploads
  • Audio tracks for audio uploads
  • Relevant text tracks for transcripts and captions.

Deliverables

• A new media content type in plone.volto
• A new media block type to provide the mentioned editing, linking and viewing functionality.
• A new content type view for displaying the new media content type
• Documentation around how to use the block for video and audio purposes, alongside the various accessibility features provided.

Risks

• Existing video blocks would not automatically get any of the new functionality without creating media content types.
• Existing users will need to be trained on how to use the new video upload experience.
• How do we provide i18n support for the transcripts and alternate versions?

Participants

Jefferson Bledsoe

[stevepiercy]

A client of mine is testing out Whisper to provide transcriptions of audio and video files used as evidence in criminal trials. It supports transcriptions and translations of multiple spoken languages, but they've only tried English. It works very well when only one voice is speaking, but it breaks down with background noise or music, or when there are multiple voices speaking at once. If no caption file is provided, and if the audio falls within Whisper's technical ability, it could provide a default transcription or translation.

As far as providing i18n support for transcriptions, I have tried Weblate with Plone core, and I am thoroughly impressed with its ease of use. See https://6.docs.plone.org/i18n-l10n/contributing-translations.html#weblate-for-translations. I'm not sure of the workflow for content.