A unified solution for <picture>

[lilith]

A Unified solution to <picture>

Perhaps there's a very simple way to support both pre- and post-layout queries with <picture>, and sacrifice neither functionality or performance.

If sources specify the dimensions of the images (and more than one image matches the media queries), we delay image fetching until CSS is downloaded; otherwise fetching can occur immediately.

We can then apply sizing constraints to further filter the list of images (media queries are still king, but if more than 1 image 'matches', we use size constraints).

I've written up the details here: https://gist.github.com/nathanaeljones/5047077

---

I also propose the expansion of the Use Cases and Requirements document to include:

11 The solution SHOULD offer an method to leverage breakpoints defined in CSS.

12 The solution SHOULD support a simplified syntax to support primary use case 3.1 (preferably a list of images and their dimensions), in order to reach users of content management systems and those without detailed knowledge of CSS media queries.

This allows complexity to be moved from HTML to CSS, and removes the need for high-volume repetition of breakpoint logic.

Authors who wish to use responsive web design will be able to use a CSS framework or snippet and matching CSS classes on to achieve responsive images - a path much less intimidating than CSS media queries, and much easier for CMSes and authoring tools to support (how would a GUI for media queries be designed)?

I fear for adoption of unless we can make it CMS and 'non-coder' friendly.

[nwtn]

The use case 11 you list could be addressed by the MQ shortcuts discussed in #39. I'd suggest adding this only if we get any traction with that.

The proposal for selecting an image based on CSS requirements is interesting. I can see what you're getting at, but I'm worried it adds too much complexity to an already complex element.

@yoavweiss based on your understanding of the how the browsers work, is something like this technically feasible?

[yoavweiss]

Picking the resources according to the element's layout means that the browser cannot fetch the resource before the layout phase was done. This prevents preloading, as well as simple loading as soon as the element is parsed, which means the practical performance of such a solution won't be any better than today's JS hacks (and possibly even worse). There are also cases when one image's dimensions are influenced by another image's dimensions, which are known only after the image was downloaded (example). That means that in some cases, the resource download will be delayed even further.

While it might be simpler for Web developers to rely on the element's dimensions, I believe the answer is better tools rather than incurring a performance penalty on the users. Better tools can be as simple as a static build step (similar to CSS preprocessors which have seen a huge success in recent years) that renders the site (e.g. using PhantomJS) and calculates the various viewport dimensions according the the element dimensions specified. The only place I can think of where this is not feasible is for 3rd party components that can be integrated in different dimensions. Some build step as part of their integration can resolve even that case.

Again, if the choice is between Web dev ease of development and the user's site performance, I believe we are obliged to choose the latter.

[yoavweiss]

@nathanaeljones Reading my previous comment, I see it may come off as harsh or dismissive, which wasn't my intention. I think your proposal provide some nice authoring tools, and may very well be easier for developers to deal with. I'm not sure about the picture{max-width:320px;} definition part, but I don't think it is necessary, as long as you define it as max-width: 100% and make sure the container is responsive (much like how we deal with liquid image layout today). I think that a "responsive images HTML preprocessor" could take your (image layout based) syntax and produce proper viewport based syntax, like the <picture>/srcset one. Would you be interested in creating such a build tool?

[eeeps]

@nathanaeljones, here's an attempt to glom intrinsic-dimension-based picking onto <picture>'s markup via javascript.

I'm repeating myself, but: we should not be thinking about rendering or marking-up content images for specific layouts, or for specific breakpoints. Images are stand-alone pieces of content, and like paragraphs, tables, lists &etc, the rendered resources and the markup that describes them shouldn't depend on a given page's layout.

@yoavweiss, hopefully some intrepid soul will enable simpler, more robust authoring via a preprocessor that generates deployed markup based on the page's CSS (backwards! that's doing it backwards!), but the goal should always be to make the core HTML/CSS/JS stack as stand-alone and human-readable/writeable as possible, no? Devs push ahead with jQuery, CoffeeScript, LESS/SASS, and the like (Flash!), and then (hopefully) those innovations (eventually) filter back to the core and we all cheer about the benifits of a standards-based, "vanilla" diet.

The comments in #39 show a willingness to toss preloading aside to allow for a certian authoring style, in certian circumstances... is this really so different?

Maybe it is. @nwtn, it pains me to say that I agree that adding another layer to the <picture> picking algorithm just feels kind of heavy, and won't fly with browser-makers who are already calling <picture> difficult-to-implement and over-specced. All of the momentum for an html (vs content-negotiation or what-have-you) solution seems to be behind viewport queries.

So I guess I'm +1'ing the proposal, while also resigning myself to the fact that it seems to be a non-starter, both with & beyond the group.

[eeeps]

Thought experiment: how would you render resources and mark-up a <picture> for a "CSS Zen Garden"-esque page that needed to take on all kinds of forms without authors being able to touch the markup?

Maybe you can get away with ballparking how important the image is, and guessing how big it will usually render relative to the viewport, given the common layout patterns of the day... "this is an 80%-width-ish image, this is a 25%-width-ish image." An inexact science, at best.

Or maybe you'd render just a couple of resources, space them REALLY far apart in size, take for granted that the image will never be rendered larger than the viewport, and use some compressive images magick? Something like:

<picture>
    <source src="1280x960-quality0.jpg" media="(min-width: 20em)" />
    <source srcset="320x240-quality50.jpg, 640x480-quality0.jpg 2x" />
    <img src="320x240-quality50.jpg" alt />
</picture>

I can't articulate why, but on first blush, that seems like the least-bad option.

[yoavweiss]

@eeeps I'm curious where on https://github.com/ResponsiveImagesCG/picture-element/issues/39 you saw willingness to toss preloading aside. The whole thread is about simplifying preloading implementation by making Web devs' lives harder (which is most probably a bad idea).

The "Zen Garden" case is an interesting one. Can you think of a realistic scenario where content images' dimensions would be different between "themes" for the same viewport? If so, any markup-only based solution will fail to satisfy this case. I'm just not sure that it's worth compromising performance for everyone for support of this (edge?) case.

[lilith]

Has anybody tried to use images inside css grid columns? Due to reflow logic, there is no correlation between viewport size and image size. In fact, as the screen gets smaller, the images get larger because the columns disappear. Media queries are useless for handling multi-column image situations. They're only useful in single-column layouts.

I just started hacking on a new vanilla js library, SlimImage that uses the true element width to decide the source image, respecting all min-* and max-* css rules throughout the hierarchy. It's <1KB gzipped and minified. Unfortunately, we're still stuck with <noscript> tags to fix the prefetch woes.

[lilith]

That said, I hope some version of <picture> manages to get implemented. There are 2 different issues I have with the current spec.

The first is shared with srcset - the red herring of pixel density. The solution to this is to offer a descriptive vs. imperative syntax.

The second is viewport-vs-context. That's a separate issue that vendors will likely never agree about. Supporting both would be syntactically simple (picture attribute "context="element|viewport"), but would make implementation more difficult.

[anselmh]

The "Zen Garden" case is an interesting one. Can you think of a realistic scenario where content images' dimensions would be different between "themes" for the same viewport? If so, any markup-only based solution will fail to satisfy this case. I'm just not sure that it's worth compromising performance for everyone for support of this (edge?) case.

@eeeps @yoavweiss In my opinion it's not. This is the same issue we're facing on device-orientation-change. We already concluded on that issue that performance is preferred over doubled download. We also considered (but never discussed further) adding an attribute for that which could "force" (but not override) browser's default setting here. video does it this way with preload attribute for example.

Has anybody tried to use images inside css grid columns? Due to reflow logic, there is no correlation between viewport size and image size. In fact, as the screen gets smaller, the images get larger because the columns disappear. Media queries are useless for handling multi-column image situations. They're only useful in single-column layouts.

While this is true a grid always refers to the viewport (if not static which is not our target). When specifying your grid you also set breakpoints based on the viewport and its initial font-size. This is the behavior of media-queries.

The same now applies for images – you set the image breakpoint based on the viewport. Of course this requires a layer of abstraction by the developer but again, this is how media-queries and RWD nowadays work.
The developer has to decide at which breakpoint (viewport based) which image will be loaded. I never had a problem with images in responsive webdesign and had no problem when using the picturefill. If I understand you correctly this seems to be a non-issue to RICG – if not, please let me know. Thanks for your input. :)

I do also think that adding an additional layer of abstraction (picture attribute "context="element|viewport") confuses developers way more than it should. By adding such attribute (which then has to be done for many more elements than just picture) you shift another layer of complexity to the developers. Regarding the hard work for many element-specs it seems nearly impossible to target this in near time. We may keep this in mind for later improvements.

[yoavweiss]

@nathanaeljones Regarding pixel density, we are in agreement, and my personal belief is that the current srcset syntax must be modified to accommodate that.

Regarding the grid situation, it is a tricky one, but unless you set the grid's width based on some percentage of its container (and indirectly, the viewport) the image's container dimensions can be changed as a function of other image dimensions coming in. That means that you either: download the images one by one or double download most of them. Also in the "one by one" technique, you'd probably have to double download since later images can impact the width of previous ones. All in all, I don't think that this particular use-case is one we should handle. (unless I completely misunderstood what you mean. If so, a gist may help)

[aarongustafson]

@yoavweiss @nathanaeljones Regarding the grid, until we get a new image format akin to SVG (which allows for media queries that are triggered based on the rendered size of the image), it is going to be a challenge for designers.

[lilith]

@yoavweiss I think it's safe to say that most containers are block elements, and therefore unaffected by their content images. With RWD, container width is generally set in CSS. In all of my websites, I've never once seen an image affect the width of it's container. I've also never seen responsive images selected based off container height instead of width (we tend to scroll vertically).

I don't see a recursive download situation as likely, and it would definitely be finite in duration due to existing browser logic to prevent the general case of relayout loops (LOTS of html elements do this, and picture would be the least of the offenders).

I use PureCSS and Bootstrap. With Bootstrap, and multiple columns, the image size is not a function of the viewport size, as it depends upon the flow hints applied to other columns (and it's own). Also, when javascript is involved (such as for my responsive carousel with multiple columns of images in each slide), additional variables are added.

Context size is not a function of viewport size alone, especially in real-world scenarios.

[yoavweiss]

@nathanaeljones An example would be extremely useful to understand your particular case.

[eeeps]

This thread seems to have sprung a few more heads since I last checked in, but...

@yoavweiss with regard to my reading of #39... I'm probably misreading? It seemed like, after you latched on to the idea of on defining MQs in the <head> as a way to keep things simple for implementers, and asked if that should be required, both @nwtn and @aarongustafson chimed in to say that no, it should not.

I don't think we have to mandate this for picture, we only have to mandate it for picture with preloading. If an author wants to have a one-off exception, I don't think it's an unreasonable tradeoff to have that exception handled outside of the PreloadScanner.

My reading: if authors don't define MQs in the head they won't get preloading. Just as they wouldn't with an optional post-layout, contextually-aware mechanism.

@nathanaeljones media queries aren't "useless" for handling grids... they're just hard to use. Unless you have some crazy script sizing elements based on the day of the week or something, element size will almost always be a function of viewport size? By which I mean: for every viewport size there is only one possible element size. That function won't always be continuous and (especially within grids) it will often be complex... and a total bummer to implicitly encode into every <picture>. Thus @yoavweiss' suggestion that we get a preprocessor to do that heavy lifting for us.

I can totally grasp that line of thinking -- better to make a few authors preprocess than to make every single user wait longer for pages to load -- but moving the web towards layout-dependent markup just gives me the heebie jeebies, I guess. Thus the Zen Garden example which I'll freely admit is more conceptual than practical.

[yoavweiss]

@eeeps - Oh, I see what you mean. That's not abandoning preloading, but possibly letting devs to opt-out. Also, that whole thread is (my) bad idea.

@nathanaeljones Found the example I had for images affecting their container dimensions (and other images' dimensions). I believe this doesn't happen often, but it's something implementations should be able to cope with, which add to the complexity of such a solution. One more thing that may complicate matters is lazy layout. If images are fetched based on layout, you're tying lazy layout with lazy loading, which is not ideal, to say the least.

[yoavweiss]

Closing this old thread for lack of any action items.