Lazy loading video element poster attribute

[asuh]

This approach for lazy loading videos is close to perfect.

<video preload="none" src="video.mp4" poster="videoposter.jpg" type="video/mp4">

In a video gallery with many videos, using preload="none" can help speed up page loading with its lazy loading capability. The one missing feature is lazy loading the poster attribute image.

I realize loading="lazy" only works for iframe and img elements, but this one enhancement to the lazy loading capabilities would potentially be a noticeable performance improvement in addition to not preloading the videos.

[annevk]

cc @whatwg/media @domfarolino

[foolip]

Paging some people involved in https://github.com/whatwg/html/pull/3752: @bengreenstein @domenic @jakearchibald @zcorpan

If we just add a loading="lazy" attribute for <video>, should it affect the poster in exactly the same way as <img src>? Or should it also do something for the video resource if no poster attribute is used?

[jakearchibald]

Gut feeling: posterloading="lazy" to control the loading of the poster, and if we want the loading of video to be impacted by viewport intersection, add another option to preload, eg preload="lazymetadata" & preload="lazy".

Otherwise, combinations like loading="lazy" preload="none" are confusing.

[zcorpan]

I wonder if it makes more sense to bring all of img's features to video's poster image by something like

<video use-first-child-img-or-picture-as-poster-also-rename-this-attribute>
 <source ...>
 <picture>
   <source ...>
   <img loading="lazy" decode="async" srcset="..." sizes="...">
 </picture>
</video>

Otherwise how do we add srcset, art direction, and type-based resource selection to poster images?

[zcorpan]

If loading="lazy" appears on a <video> I would expect it to apply to the video, or maybe both the video and the poster. Its behavior is orthogonal to preload in that preload specifies how much to load, and loading specifies when to load (ASAP vs wait until it's in view or almost in view).

[jakearchibald]

I wonder if it makes more sense to bring all of img's features to video's poster image by something like

I like that!

…loading behavior is orthogonal to preload…

loading="lazy" preload="none" seems really weird though

[zcorpan]

It's not so weird if it also applies to poster="...", in my opinion. I guess the loading attribute would have no effect when there's no poster attribute and you use preload="none", sure. Is it a problem? loading="lazy" with src="data:..." is also weird, but isn't disallowed.

[zcorpan]

I think these are the options to solve this issue:

• Add a posterloading=lazy attribute to control lazy-loading of the poster image (but not the video).
• Add a posterelement="#imgid" attribute to point to an <img id=imgid> element to use as the poster image, and support (almost) all of <img>'s features also for poster images. This also solves other feature requests like picture/srcset support for poster images. See #4004
• Don't add any new attribute, but use lazy-loading behavior when preload=none and dimension attributes are specified, as suggested in https://bugs.chromium.org/p/chromium/issues/detail?id=1317849#c6 (personally I don't like this option, as noted in the bug).
• Add a loading=lazy that controls both the poster image and the video with the same attribute.

For lazy-loading of the video:

• Also add loading=lazy to control lazy-loading of the video (but not the poster image).
• Reuse the preload attribute to control lazy-loading of the video (but not the poster image).
• Add a loading=lazy that controls both the poster image and the video with the same attribute. (This is also in the above list.)

[zcorpan]

I'm not sure it's worthwhile to specify lazy-loading for video, since preload="none" exists and people can use lazy-loading poster images (when that is supported).

[scottaohara]

@zcorpan would

I wonder if it makes more sense to bring all of img's features to video's poster image by something like

mean that mean authors would be able to provide alternative text for the poster then? because that'd be something people have wanted for quite some time, and would be most welcome.

[zcorpan]

@scottaohara maybe, but I think it's better to discuss alt text (or accessible name) for video (and audio) in a new issue.

For now, aria-label or aria-labelledby can be used, right?

[scottaohara]

We're talking about different things, @zcorpan. Those attributes can be used to provide a name for the 'video' element, but they would not (and should not) be used to provide a description for the poster itself. This graphic is often used (misused) to provide some sort of title card with content that is only available via that image, and is only relevant when that image is displayed, has long been a frustrating topic for some.

Here is a closed HTML AAM issue which then links to other past issues on the subject. (following the links leads to some pretty heated 'discussion')

Understood if you think this should be spun out into a separate issue, but the reason I brought it up is because of your mention of "all of img's features", and if this idea is accepted, then this image (and thus its alternative text) could be exposed.

[asuh]

Here's what I'm reading from the various conversations above, also with my preference.

<video
  src="url_of_src" // URL of video source
  poster="url_of_img" // URL of image source
  aria-label="Video element name" // Describes the name of the video element, similar to describing a button label
  alt="Description of video content" // Describes the contents of the video itself, similar to image alt text
  loading="lazy" // Lazy load the video element and poster image
  preload="none" // Don't preload the video element, similar to loading="lazy", but could also help define how to lazy load the video
>

It makes more sense for loading="lazy" to be a consistent attribute name with consistent functionality. Even if the loading attribute duplicates most of preload, I would always be inclined to add loading to all media for better performance. Education can help differentiate it from preload and when/where to use either.

[zcorpan]

@scottaohara ok. Yeah, I think a separate issue would be best to align on how to expose the poster image to ATs. It might inform how we want to solve this issue, but still lazy loading and accessibility are orthogonal concerns.

[zcorpan]

I think we should add posterloading=lazy to solve this issue, since it's least complex and the other things can still be added separately (lazy-loading the video with loading=lazy, adding alt text for the poster #7954 could be a separate attribute like posteralt, and srcset/picture support for posters I think should be solved together with #6363).

[zcorpan]

PR: #8428

This issue still has needs implementer interest. @yoavweiss is there interest for Chromium? @annevk for WebKit? @emilio for Gecko?

[annevk]

posterloading seems like a reasonable addition, yes.

[yoavweiss]

^^ @chrishtr for Chromium interest, as his team is likely to own the implementation work.

[chrishtr]

Confirmed interest from Chromium.

[zcorpan]

Thanks @annevk @chrishtr !

What's remaining is review of #8428 and the tests.

[emilio]

This seems fine to me as well. In general it'd be great to have it better-defined how <video poster> works (in Gecko the poster is just an anonymous <img> element), but that might be a bit trickier... It could unlock very cool stuff tho, like something like:

<video ...>
  <picture slot="poster">...</picture>
</video>

or pretty much anything (doesn't have to use the slot attribute, could be magic a la <details>, but you get the idea...).

The shadow tree of <video> could be specified as something like:

#shadow-root
  <slot name=poster></slot>
  <ua-defined-controls></ua-defined-controls>

Where the ua-defined-controls contents are, well, UA-defined, and would include captions, the video controls, etc.

I wouldn't oppose to the one-off extra attribute, but that would be perhaps a more flexible / better design over-all... Would there be appetite for such a thing?

[zcorpan]

Thanks @emilio

That would be like the second option in https://github.com/whatwg/html/issues/6636#issuecomment-1125404102 but without ID lookup (instead relying on picture or img being a child or descendant of video + some opt-in).

It's more work, but we get more features. Curious to hear what others think.

[johannesodland]

I wonder how this would look if the source media attribute was reintroduced later as in #6363.

[scottaohara]

It's more work, but we get more features. Curious to hear what others think.

just chiming in again that the idea of using picture > img within a video would be welcome for providing alternative text for the poster image. https://github.com/whatwg/html/issues/7954

[annevk]

How common is poster? A bigger redesign might be worth doing if it's popular and also if there are already popular libraries available that address some of the shortcomings.

[zcorpan]

Web Almanac: https://docs.google.com/spreadsheets/d/1T5oVAVmcH3sM6R-WwH4ksr2jFtPhuLXs3-iXXoABb3E/edit#gid=1515756004 - though poster appears multiple times and I'm not sure how to interpret pct_attribute and pct_videos. @eeeps can you help?

Chromium use counter V8HTMLVideoElement_Poster_AttributeGetter is at 2.1123% and V8HTMLVideoElement_Poster_AttributeSetter at 2.0306%, but I assume this doesn't count use in HTML without author JS accessing the getter/setter, and if so, usage is probably higher.

A quick query in httparchive sample_data (10,000 pages) gives 445 pages = 4.45% of pages, and 1743 pages use video, so 25,5% of pages with a video element also use poster.

queries

``` SELECT COUNT(DISTINCT page) AS num FROM `httparchive.sample_data.response_bodies_desktop_10k` WHERE REGEXP_CONTAINS(body, r'(?i)(]*\sposter\s*=)') ``` 445 ``` SELECT COUNT(DISTINCT page) AS num FROM `httparchive.sample_data.response_bodies_desktop_10k` WHERE REGEXP_CONTAINS(body, r'(?i)(

[kevinfarrugia]

Incidentally I was looking at a similar query and the results are similar to what you have in the sample data. Uses the same data set as the Web Almanac 2022, (June 2022) sampled at 10%.

+---------+-------------------+-------------------+-----------------------+-------------+
| client  | pages_with_videos | pages_with_poster | pct_pages_with_poster | total_pages |
+---------+-------------------+-------------------+-----------------------+-------------+
| mobile  |             38847 |             10458 |    0.2692099776044482 |      771291 |
| desktop |             35098 |             10071 |   0.28693942674796286 |      558537 |
+---------+-------------------+-------------------+-----------------------+-------------+

pct_pages_with_poster only includes pages which have at least one video element.

Query

```sql #standardSQL CREATE TEMPORARY FUNCTION getAttributes(payload STRING) RETURNS STRUCT LANGUAGE js AS ''' try { var almanac = JSON.parse(payload); var obj = { poster: 0, }; for (var node of almanac.videos.nodes) { if (typeof node.poster != "undefined") { obj.poster++; } } return obj; } catch (e) { return {}; } '''; SELECT client, COUNTIF(total_video > 0) AS pages_with_videos, COUNTIF(attributes.poster > 0) AS pages_with_poster, COUNTIF(attributes.poster > 0) / COUNTIF(total_video > 0) AS pct_pages_with_poster, COUNT(0) AS total_pages FROM ( SELECT _TABLE_SUFFIX AS client, SAFE_CAST(JSON_EXTRACT_SCALAR(JSON_EXTRACT_SCALAR(payload, '$._almanac'), '$.videos.total') AS INT64) AS total_video, getAttributes(JSON_EXTRACT_SCALAR(payload, '$._almanac')) AS attributes, FROM `httparchive.pages.2022_06_01_*` TABLESAMPLE SYSTEM(10 PERCENT) ) GROUP BY client ```

[zcorpan]

I discussed with @chrishtr, he said supporting picture might be too much complexity. But supporting img still gives srcset, alt, and other features of img.

@emilio wrote

<video ...>
<picture slot="poster">...</picture>
</video>

or pretty much anything (doesn't have to use the slot attribute, could be magic a la <details>, but you get the idea...).

Yeah, I think we shouldn't use slot for an HTML-native feature. We could use a new boolean attribute on video:

<video posterchild>
  <img ...>
  <source ...>
</video>

(Please suggest other names for posterchild, although I also kinda like it 🙂.)

[annevk]

Does <video> need an attribute? Is an <img> child not sufficient?

[johannesodland]

I discussed with @chrishtr, he said supporting picture might be too much complexity. But supporting img still gives srcset, alt, and other features of img.

Given the resent reintroduction of the media attribute on media element sources, do you think it's worthwhile to reconsider the picture element? I'm concerned there might be an inconsistency if the video element is able to support art-direction for videos but not for the poster image.

[zcorpan]

@annevk it might change behavior for legacy web content that already use img in video intended for old browsers without video support.

[jernoble]

Why not a new <poster> element, defined to have the same semantics and behavior of <img>?

Non-video supporting browsers would ignore it. Video-supporting browsers without specific support for it would ignore it as well.

[zcorpan]

But it would only do something when it's a child of a video element?

[jernoble]

Sure. Much like adding a poster= attribute on any non-video element doesn't do anything.

[jernoble]

(<poster> could also be a <picture> element instead of a <img>. The goal with this straw man was to solve the problem of legacy, non-video supporting UAs.)

[zcorpan]

I don't worry about what happens in non-video supporting UAs, most likely most of the web doesn't work in those browsers for other reasons anyway. I'm more concerned about currently existing web content and the effect for such content in new browsers. However, I haven't looked into how common it is to have an img in a video; if it's very rare maybe we can do what @annevk suggested in https://github.com/whatwg/html/issues/6636#issuecomment-1554502038

[arogers-enthink]

Art direction [out-of-the-box] would be a big plus when supporting prefers-reduced-motion.

[scottjehl]

I like the sound of picture being used inside video to define poster images, since it'd support the variety of conditions that video source elements can now target via their media attributes, and offer srcset and sizes as well. In that case, would the picture element need a child img element or would the picture element act as a controller for the source of the video poster attribute's currentSrc?

Edit:

I suppose a child img would offer the loading=lazy behavior, so it's nice to have there.

That said, if video itself offered loading=lazy, could that attribute determine how other video features behave in addition to poster? For example, could loading=lazy combined with autoplay cause a video to delay its data fetching and autoplay behavior itself until the video element is in (near) the viewport? Lazy autoplay is another feature I'd love to see addressed declaratively and it'd be great if this one attribute we already know could drive it. (related new issue: https://github.com/whatwg/html/issues/10376 )