Video Block: alternate sources such as ogv and webm file formats

[Soean]

Description

Video blocks should have the ability to add other file formats as alternate sources.

Classic Editor:

[melchoyce]

Took a stab at designing this within the block sidebar:

[afercia]

Not sure the alternate sources should go in the sidebar. These are not settings. They're more related to content and the act of editing. As pointed out in the accessibility team report, one of the main accessibility barriers in Gutenberg is the extremely difficult keyboard navigation between the content area and the sidebar.

Ideally, while editing the blocks, the sidebar should be used only for very minor settings. Anything content related should be in place, within the block.

[melchoyce]

Yeah, that makes sense. In the interim, what's the possibility of having an edit button in the quick toolbar that opens the media library, so we can use the existing settings screen?

[afercia]

Media views/modal/library have their own accessibility issues but from a technical perspective I guess it's feasible? Best persons to ask: the media component maintainers 🙂

[melchoyce]

I tried an idea where both video formats and available within the block content, but it felt really overwhelming: https://cloudup.com/cawhmW_i7KV

So, I tried a version where these settings are available via the Toolbar:

Selecting "add more formats" would open a browser upload dialogue, and once you select your formats, they'd show up underneath the video inside the block:

This feels more approachable to me — it's less overwhelming, and the settings are available when you need them, versus always taking up your attention. @afercia, would this solve the issues you brought up?

[afercia]

Discussed during today's accessibility bug-scrub. From an accessibility perspective, probably the two most important things are:

• alternate sources
• captions/tracks

Both should be prominent and ideally not "buried" within the more menu, where they're not so discoverable. We'd suggest to try an always visible control "Add alternate sources" that toggles the UI. Also, a short explanation about why alternate sources are important may help. A sort of description is displayed in the current media views modal, that can be certainly improved :)

[afercia]

Noting that this issue was reported in the WPCampus accessibility reports, see #15281, and as such should be moved to the related project in the "To do".

[youknowriad]

This was discussed today in the #core-editor triage session (link requires registration):

https://wordpress.slack.com/archives/C02QB2JS7/p1557753241107800

Based on the reading and the discussions, the current plan is:

• Add a button to the block toolbar "Add alternate source"
• Clicking the button opens the media modal to upload/select alternative video sources
• When the block is selected, show the alternate sources https://user-images.githubusercontent.com/2846578/51499357-f50c4500-1d97-11e9-8d65-9d9b9d193550.png
• Update the output of the block (save function) to include these sources

The design and the flows can be iterated on but this seems ready to be explored.

[Soean]

That sounds good. I'm gonna keep working on that ticket.

[drzraf]

Is this only about alternate formats or integrates alternates sources (like in "same video hosted by another video-provider*). I think hosting at multiple providers (eg: Youtube&Vimeo, Dailymotion&Peertube) should be considered as part of alternates sources (as long as such a source is supporting oEmbed).

[mapk]

To make sure we have a unified mockup of the next steps, here's one based on @youknowriad's comment and @melchoyce's earlier mockups.

[drzraf]

@melchoyce : Wouldn't Selecting "add more formats" would open a browser upload dialogue, but also provide a way to select remote URL too?

[afercia]

To make sure we have a unified mockup of the next steps, here's one based on @youknowriad's comment and @melchoyce's earlier mockups.

In the above mockup, the buttons visible text represents the current value (the source file name). As pointed out in many related issues, buttons and generally all UI controls should communicate the available action and not the underlying selected value / option.

The current interface in the media views does it pretty well and shows:

• the format type
• the source file name (and path)
• a separated "Remove video source" button
• the available alternate source types (if any)

Comparing to the current UI, I'd tend to think in the new proposed one there's a loss of useful information and reduced usability / accessibility. Moving controls within the block UI shouldn't happen at the cost of a reduced user experience. I'd like to recommend to think at an improved UI, one that preserves all the current features and information.

[mapk]

Adding so many inputs and options to the video block seems a bit much. They're also problematic in a popover of sorts. These alternative sources feel more like secondary actions to me. For these reasons, alternative sources for the video might actually work a LOT better in the sidebar.

I'm a bit unfamiliar with how alternative sources work, but does each alt source show up on the frontend? If not, all the more reason we move these to the sidebar and keep the block as close to 1:1 with the frontend.

[melchoyce]

The alternative sources are fallbacks that only load if your browser doesn't support the primary video format uploaded, I believe.

[land0r]

Any news? Because this feature is an urgent thing, don't understand why you do not add alternative sources "from box". Unfortunately, Gutenberg still so raw development and editor tool

[mapk]

@enriquesanchez, do you think my comment here in regards to using the sidebar is a viable accessible option?

[paaljoachim]

I think it would be good to talk about this issue during todays Core-Media chat. As it (to me) seems like there is agreement to place the needed options in the sidebar. Perhaps we could also aim for getting this included into WordPress 5.4.

[enriquesanchez]

do you think my comment here in regards to using the sidebar is a viable accessible option?

@mapk I believe so. Navigation to and from the sidebar has been improved recently (see https://github.com/WordPress/gutenberg/issues/13663).

I agree that having these options in the sidebar is our best path forward for now. I think Mel's mockups and the examples from the media views could fit well in the inspector.

[afercia]

Can we discuss this in the next accessibility meeting please? I guess not everyone in the team would agree navigation to and from the sidebar has really been solved.

[sabernhardt]

Is it possible/feasible to group video files in the Media Library instead of just within the post editor (or a widget)?

If so, then those alternate formats theoretically could be combined in all situations:

1. the video's attachment page
2. another post/page
3. a video widget

And if the formats can be grouped, then we might also want to connect a poster image and subtitles (or any <track> files) to the video attachment group.

Context-specific settings such as looping and muting the video would still need a place to edit them for each situation, perhaps within the sidebar.

Visually, this might look close to the current embedding workflow with Classic Editor, or maybe something more like the "Add Subtitles" button concept here: https://github.com/WordPress/gutenberg/issues/7673#issuecomment-515607606

[mapk]

I think I may have missed what the Accessibility Team discussed in the meeting. Was it discussed, @enriquesanchez?

[enriquesanchez]

@mapk it was discussed and notes from the meeting can be found here and a follow up discussion here.

To clarify, the discussion focused on the accessibility of the sidebar and not on this specific issue:

The team had a very productive discussion that addressed the longstanding accessibility issues, recent improvements, and suggestions for solutions.

The team agreed to work together on a proposal that described the problem, suggests needed structural changes, and provides visual concepts.

Once 5.4 Beta is out, @joedolson, @afercia, and @nrqsnchz will collaborate to create the proposal.

I still stand by my comment above and think that while not perfect, navigation to and from the sidebar has improved considerably.

[MikeNorman]

Hey folks!

Dropping some feedback in here that may be relevant.

I tried to put a Droplr link into the video player and didn't get an error message it just didn't work https://d.pr/v/2d3wZg

and when I dimply tried pasting this link into the P2 comment I received the following error https://www.loom.com/share/0df32fdf24804f5b9568d0ba0670daae https://d.pr/v/fQ5Wfk

[enriquesanchez]

Started looking at this issue again after working on #7673 and I'm wondering if a similar flow as the one proposed there can also work here:

The option to add alternate sources appears in the toolbar:

Assuming we can detect the video file format, once the user uploads an alternate source we display the file in the same popover:

Once/if all possible file formats have been added, we remove the prompt to upload alternate sources and only display the files:

---

I do have some questions I'd like to clarify and that may help inform the best flow for this issue:

1. WordPress supports a variety of video formats. Why do we only ask for mp4, webm and ogv in the Classic editor?
2. If I upload any of those formats, do I get the option to upload the remaining two? So say I upload an ogv file, that means I'll be prompted to upload an mp4 and webm versions?
3. Do we do any file extension check? I wonder if the answer is no and that's why in the Classic editor we have two separate add buttons for each alternate format.
4. What should happen if I 'Replace' the video? Do the added alternate sources get removed as well?

[afercia]

@enriquesanchez this seems very promising to me.

WordPress supports a variety of video formats. Why do we only ask for mp4, webm and ogv in the Classic editor?

Honestly I don't know. It's a question the Media team can answer better than anyone else. It might be that either the documentation page or the media views (or both) haven't been updated in a while.

Re: the other points, I'd Cc @joedolson as he's much more expert in Media accessibility,

[kellychoffman]

I think this is a good iteration but could use further exploration.

• The current separation of what goes into the sidebar and in the block toolbar feels a bit random. (Why do you add the poster image in the sidebar but the caption file in the toolbar, for example?)
• These don't feel like "top level items" and adding icons for each new item will not be scalable
• The icons are not clear
• Unclear why "replace" written out and alternate sources are added as icons

[afercia]

The current separation of what goes into the sidebar and in the block toolbar feels a bit random. (Why do you add the poster image in the sidebar but the caption file in the toolbar, for example?)

I'm a bit biased (accessibility guy here) but if you ask me well I'd say that any block setting should be displayed "in place" close to the block as the jumping to the sidebar is a huge accessibility barrier (discussed at length in this project over the last 3 years).

More realistically, "secondary" settings can go in the sidebar block "inspector". More important settings and settings related to content creation, like alternate sources are, should go in the block UI, in my opinion.

[joedolson]

@kellychoffman As I see it, the caption file is critical content, so it should be edited with the content, where the poster image is largely ornamental. If you feel those should be managed in the same area, however, I'd have no serious problems with editing the poster image within the block toolbar. However, I'd consider moving the captions to the sidebar unacceptable.

Regarding the clarity of icons: I'm certainly not somebody who will argue that icons aren't clear. I'd be absolutely thrilled if these icons were eliminated in favor of text. I have no opinion on the icons themselves, particularly, as - frankly - I don't generally get much meaning out of icons, and am dependent on the tooltips to understand what most buttons do. But always happy to remove icons in favor of text.

[cricalix]

Hello. It's been a year since the last update on this work - has any progress been made that brings Gutenberg to parity with the older editor?

[Soean]

@cricalix No progress so far.You can start working on a solution if you like.

[porg]

1) 👍 My upvote for this function.

2) 👉 In my followup comments (one by topic) I will place:

• UX design critiques on some of the mockups / comments.
• Polls regarding requirements and feature scope.
• If I receive enough reactions I may at a later time provide mockups which address all these together.
• But at least wanted to share my concerns and get some discussion/input.

[porg]

Concept "Buttons per container format" by @melchoyce et al

Summary: Buttons per container format, once that file type is added, the corresponding button gets inactive.

• Solution does a) not scale well and b) is conceptually wrong for these reasons:

  • Container format says nothing about the audio/video codec.
  • Users may want to provide an MP4 with a H.264 (avc1) codec and another MP4 with a H.265 (HEVC) codec. Which can be stated with the additional attribute codecs in the <source> element of a <video> element.

  <video muted playsinline autoplay>
      <source src="clip.webm" type="video/webm; codecs=vp9">
      <source src="clip-hevc.mp4" type="video/mp4; codecs=hevc">
      <source src="clip.mp4" type="video/mp4; codecs=avc1">
  </video>

[porg]

Goal Sources always visible on canvas from accessibility meeting

1. Am strongly against a UI implementation like in mockup by @mapk with the video sources as buttons below the caption and the video. UX design rationale:

   • The sources are not visual content but are metadata.
     • Showing them as visual extensions to the video block probably destroy layouts (think: video tiles, cards, etc)
     • Or at least severely challenges/distorts layouts.
   • The Block Editor should be as WYSIWYG as possible
     • Among its main design goals!
     • Needing to resort to the Frontend "to get an impression" should be the absolute exception (only acceptable for very dynamic/runtime stuff)

2. Though I agree we must aid discoverability and also indicate on canvas that a video block has multiple sources. Ideas:

   • Toolbar: Change Replace button to Video source(s): 4 [✏️]
   • Responsive badge superimposed over videoblock in one of the corners
     • Minimal version just a number like (4)
     • If there's more space available it can also be (amount) + shortnames of the formats 4: WEBM, MP4, OGG, …

[porg]

Poll: Set source order (=priority) — Is this a design requirement?

In the HTML output the order of the <source> elements within the <video> element matter. The web browser considers them top to bottom, taking the first one which satisfies its media requirements. Vote

1. ❤️ Hardcoded by WordPress as a platform — Following the guideline Decisions, not options
2. 🎉 Per-site preference at: WordPress → Settings → Media → Video → Video source order
3. 🚀 Per video order
   • If this is identified as a user need and hence design requirement, then the solution could be like this:
     • ☑︎ Use globally set source priority
     • That option is integrated into the inspector (yet another reason NOT to do this on canvas)
     • Option is ON by default.
     • If toggled OFF, the source filepath fields get drag handles to be re-orderable to reflect your source priority.

[porg]

Poll: Set feature scope: Video block shall also support…

1. 🎉 Different sized sources — That is same format, same codec, but different sized videos.
2. 🚀 Adaptive Streaming like DASH
3. ❤️ None of the above — For these needs better use YouTube, Vimeo or a 3rd party video plugin like Foliovision.

[drzraf]

The subject is "alternate formats" while what is actually discussed is "alternate sources" and I believe "alternate remote sources" is part of it. I wish core/embed supported multiple <url, type> tuples attributes (or a new sources Array attributes)

(I'd like to mention here https://github.com/WordPress/gutenberg/issues/20105 which is about 3rd-party video service fallback)