Empty source set should be allowed

[yoavweiss]

While talking to @colinbendell, he mentioned a use case he has: loading certain images only on some viewport sizes, but loading nothing on others. My response was that this should be possible by eliminating the srcset attribute from that <source>, but it doesn't seem like we specced it that way. Unless there's some other way to answer thzt use case, we probably should.

Can we modify https://html.spec.whatwg.org/multipage/embedded-content.html#the-img-element:attr-picture-source-srcset-3 so that a lack of srcset would result in a returned empty source set?

I hesitate regarding 5 there. What should we do if there are no candidates?

[zcorpan]

I think this was intentional to have a story for future extensions.

Can you elaborate on the use case?

[yoavweiss]

As I understand it, the use case would be a Web site where the wide viewport breakpoints includes certain content images that are not present in narrow viewport breakpoints.

So, images that today have a display: none rule on them for certain breakpoints, and are downloaded anyway.

[zcorpan]

Interesting. How about we return empty source set if srcset is absent, but continue the algorithm if srcset is present but does not result in any parsed image sources? That would satisfy the use case and have the same forward compat story.

[zcorpan]

Although since srcset parsing mostly ignores unknown descriptors, it seems like it won't return an empty source set anyway for future extensions to srcset. So maybe we can stop the algorithm when srcset is present also...

[zcorpan]

Another issue here is that if the make the algorithm return an empty source set, I think that will result in an error event being fired and alt text showing up (or broken image icon). I guess that's not quite the wanted result.

[yoavweiss]

Another issue here is that if the make the algorithm return an empty source set, I think that will result in an error event being fired and alt text showing up (or broken image icon). I guess that's not quite the wanted result.

It's probably something authors can live with (display: none the alt text and ignore the error), but it would be better if we could avoid that altogether.

[zcorpan]

Yeah. So what is the ideal behavior? Fire load event? Fire a new event emptied? Should the rendering be display:none? Or like <img alt="">?

[zcorpan]

I think this is blocking on https://www.w3.org/Bugs/Public/show_bug.cgi?id=26702 for a proper fix.

[Wilto]

Ah man, I am super not into the idea of showing/hiding an image completely using picture. I know requests are complicated when it comes to display: none on an image and parents thereof, but we don’t encourage people to strip src off img when they want to hide one. Besides, an image that’s optional at some sizes is likely in “decorative” territory anyway, so it might be a better job for CSS from start to finish—hiding real content “for mobile” is a pretty well established not-best-practice with RWD.

-1 to changing behavior to accommodate this.

[yoavweiss]

I was under the impression that we have discussed it in the past as a valid use case, but maybe I was hallucinating...

I guess that authors can do this by setting the src to some 1x1 transparent GIF, that would expose this as the hack that it is.

[zcorpan]

@Wilto's argument that images that are hidden in some layouts are probably not content images seems compelling. But it's hard to know without having real examples to study.

Moreover, images that are hidden in some layouts seem like they will not be critical in the layouts where they are visible, either, so it should be fine to use lazyload (or whatever when it exists), which will not download anything for display:none.

So lacking examples and the fact that it's easy to work around (don't even need 1x1 GIF, just use data:,x) makes this WONTFIX for now at least.

[yoavweiss]

@aarongustafson - Can you share real-life use-cases for this?

I'd prefer not forcing authors to got through data-URI copy-pasting for valid use-cases.

Reopening, since I think we need to discuss this some more.

[attiks]

FYI: We had similar request for our Drupal 7 picture module and we added an option so people could select 'empty image', in the back we output a data-URI, but I agree doing this by hand is optimal.

I think their main use case was to be able to hide certain image fields on smaller viewports.

[aarongustafson]

I am working on the use case piece and will post here as soon as it's ready.

On Wednesday, March 11, 2015 at 12:47 AM, Peter Droogmans wrote:

FYI: We had similar request for our Drupal 7 picture module (https://www.drupal.org/project/picture) and we added an option so people could select 'empty image', in the back we output a data-URI, but I agree doing this by hand is optimal. I think their main use case was to be able to hide certain image fields on smaller viewports.

— Reply to this email directly or view it on GitHub (https://github.com/ResponsiveImagesCG/picture-element/issues/243#issuecomment-78218259).

[BenRacicot]

I had opened up a SO question about this very thing and have ended up here. I believe the best use case is that on some devices (mostly mobile) you will want to load/parse/cache no image. This would not be a display:none; situation. It would need to be an <img src="">.

My argument was that if the picture source for a specified size was empty it should load no image for those devices.

[zcorpan]

@BenRacicot thanks. You say what your wanted behavior is, but I also want to know why. Is it only to save bandwidth? Is it only to save space? (If the former, why not show the image if it's already cached?)

Can you elaborate on why "This would not be a display:none; situation. It would need to be an <img src="">."? Do you want alt text to show up? Broken image icon?

[BenRacicot]

Hey @zcorpan , Yeah so my goal is to be able to save bandwidth on mobile's, as of now there is no way to do so. Obviously display:none; still downloads the image.

Once an empty img src loads, we could then define via CSS if the img tag itself is display:none; to hide alt text etc.

I found it surprising that this isn't even an option right now.

[zcorpan]

@BenRacicot OK, it seems you are reasoning around the current behavior of only setting display:none. But we're here to change the spec. We can make no image be loaded and hide the image (as display:none), if that is the behavior that we want. Or not hide the image by default, if that's better. This is why I want use cases and actual examples.

Also see https://github.com/ResponsiveImagesCG/ri-usecases/pull/57#issuecomment-88866121

[BenRacicot]

Ah I see. Yes a UI Developer I am sometimes tasked with site speed. Loading smaller images on mobile devices is great for reducing size but sometimes actually omitting content on these devices is preferred.

So regularly I believe developers would utilize loading nothing on mobiles, a medium sized image on tablets and large on desktops.

Please let me know how else I can help.

[COLABORATI]

It is not clear for me how this discussion ended and I can not find any hints in the docs about how to NOT load images on e.g. smaller screens. This is a very common use case, did this really find no way into the specs?

[BenRacicot]

@COLABORATI I agree. All went quiet exactly 1 year ago. I'm still very surprised that loading no image on mobile is not part of the specification for <picture> or srcset. Since it is not, argument can be made that these tags are more meant for artistic direction than performance because if performance was a factor they've left out the most common performance gain on mobile, which would be loading no image.

If anyone comes across some information or how we can influence the spec in this regard I would love to give my time to this. Please post back here.

[COLABORATI]

OK, wait - I want to be really sure that I am not misunderstanding things here. So is it correct to write: "Currently the responsive image specs include no way to not load images on smaller screens" ?

Yes, I found the SO question mentioned here, that feels a little hacky, but it seems to work. However, an official way of doing that would be nice. Such a long process and they really missed that?

[zcorpan]

Yep. Lots of things are missing! Baby steps, we can't do everything at the same time. This use case was not considered a requirement (more "nice to have"), and e.g. @Wilto above disagreed we should have it at all. Maybe the time is ripe now to try again, though; it's not too late to add things. I recommend checking if an implementor is interested in working on it (we have @yoavweiss in this thread but can also file bugs/feature requests for Gecko and MS Edge).

[COLABORATI]

I see no logic in the argument of @Wilto at all - in a normative discussion it does not matter if a picture is considered "decorative" or not - the discussion should be based on facts and requirements, not speculations about the designers decision process.

Fact is that it is a good recommendation to load as few pictures as possible for mobile users - but of course on a full hd desktop screen everybody likes to show a lot of great pictures. So the requirement that follows is to have a way to set "no picture" as an alternative source for the image tag.

I am not trained in how to write technical specs, but I hope it is clear what this is about. Please ask, if not.

[yoavweiss]

I'm in favor of implementing this assuming compat and back compat is not at risk.

@benracicot: can you share a site (or several) that currently use the data uri hack? (So we can have a better understanding of the use case)

[aFarkas]

@yoavweiss

I'm not sure what the plans are, but I would say compat is at risk, if this is done without any special boolean attribute.

If you think of lazysizes you might have the following markup:

<picture>
    <source data-srcset="img-candidate" media="(matches)" />
    <img />
</picture>

Now we have a matching source without a srcset. As I understand this issue correctly, in this case the image should become display: none?

[yoavweiss]

No one's discussing implicitly setting display: none afaict. I don't think that the above case poses a backcompat risk, as no resource will get picked either way.

A case where <source> had no srcset and the expectation is that it will be skipped (and a <source>/<IMG> will be picked) will pose a risk. Not sure that case is prevalent (never seen one)

[BenRacicot]

@yoavweiss I believe it was @attiks that has a data-attr hack-fix with their Drupal module.

I agree with your above backwards compat case. It seems that since this thread's proposed 'empty src loads no image' was not the original implementation the behavior currently does load the default img tag's src image. See code below or this plnkr

<picture>
    <source srcset="http://placehold.it/1000x150" media="(min-width: 1000px)">
    <source srcset="http://placehold.it/800x150" media="(min-width: 768px)" alt="this loads on mobile">
    <source srcset="" media="(min-width: 0px)"> <!-- this should fire on screens between 0 and 767px -->
    <img src="http://placekitten.com.s3.amazonaws.com/homepage-samples/200/140.jpg"> <!-- this should only load when picture is not supported -->
</picture>

1) Would anyone agree that the proper behavior of <img> with an empty src would be to 404 the image? That being said, and just for reference, the <picture> spec says:

...As well, the picture element itself does not display anything; it merely provides a context for its contained img element that enables it to choose from multiple URLs.

So, maybe I'm misunderstanding the above but, shouldn't the picture element source behave the way native <img> tags work? Which would at least 404 an empty source.

2) Also, there shouldn't be a backwards compat issue since no one should have empty source srcset's in their markup right now.

3) I believe that CSS does a play a part where if the img tag does load on a specified screen with no image src we would hide it with CSS. However CSS is not the place for image loading performance because of how image URL's and data URI's are loaded and cached.

4) Another use case are countries which have high cost of bandwidth and bad internet connection because of bad practice of ISP's and limited access all together, such as Africa. A developer building a website where majority traffic comes from one of these areas would no-doubt want to remove images from mobile versions and possibly every version but desktop.

Does any of this help?

[aFarkas]

@yoavweiss @zcorpan brought up the question:

Yeah. So what is the ideal behavior? ... Should the rendering be display:none? Or like <img alt="">?

[zcorpan]

1) Would anyone agree that the proper behavior of with an empty src would be to 404 the image?

I'm not sure what you mean by "404 the image". But the correct behavior for <img src=""> is to not load anything and fire an error event.

I wouldn't mind adding a new attribute like nosrc to source to opt in to this behavior (and maybe honor it even if srcset is also specified, to provide a data: URL GIF for current-era browsers). I suggest rendering it the same as <img alt=""> (so it can be styled e.g. to take up some space) and firing a new event (hide?).

[jeremyzahner]

@BenRacicot

Would anyone agree that the proper behavior of with an empty src would be to 404 the image? That being said, and just for reference, the spec says:

Actually, since if there is no image present, no network request would be made. Since 404 is a http networking status code (you could say) we are not talking about 404 here in any way.

[jeremyzahner]

Ah man, I am super not into the idea of showing/hiding an image completely using the picture. I know requests are complicated when it comes to display: none on an image and parents thereof, but we don’t encourage people to strip src off img when they want to hide one. Besides, an image that’s optional at some sizes is likely in “decorative” territory anyway, so it might be a better job for CSS from start to finish—hiding real content “for mobile” is a pretty well established not-best-practice with RWD. -1 to changing behavior to accommodate this.

@Wilto Makes an entirely valid point. By all meanings, we should probably respect the picture elements presence as what it is: a picture. That (at least to me) means, that network traffic itself can be no argument on changing the spec for that particular element. So, in my opinion, this discussion should be about browsers loading sources for images that are not displayed (aka: the display: none; problem). By sound logic, an element (like a picture) is either visible/present/displayed or not. That should be the actual entry point for a browser to determine if an image source should be loaded or not.

[aarongustafson]

I would agree with @Wilto that we should never hide real content from folks, but then there’s the judgement of when is image content nice-to-have vs. integral to the content.

The case I’d give is a (relatively) common one: a newspaper homepage. On a large screen where there is a lot of competition for your eye, using thumbnails can draw your eye to the most important story teasers, but that is an affordance, not content. (In most cases you don’t get a whole heckuvalot out of those tiny thumbnails). On a smaller screen, there’s little to no competition between article teasers because they are often presented linearly. Order typically indicates priority in those instances and the images are superfluous (and potentially undesirable from a performance standpoint). In order to meet this need, we currently need to use JavaScript to lazy load in the img (or, better yet, picture) when the available screen real estate (and therefore layout) makes it advantageous to have them. Maybe that’s the best way to handle it—I’m not sure—but it would be nice to have declarative markup to enable the same approach without the additional dependency. Currently all we’ve got it data-* and JavaScript.

[RicoBrase]

Agreeing with @aarongustafson.

I'm from germany, it's actually very common here, that our mobile data plans have a limited volume. If that is used up, our speed is reduced from 4G/LTE/whatever to about 16-64kbit/s (depending on the provider). Therefore it would be actually quite useful to stop loading images right away.

I would prefer using an optional no-src attribute:

<picture>
    <!-- sources for visible images -->
    <source srcset="" no-src ...>
</picture>

The default behaviour: displaying nothing, as if there is no element at all (css display: none;), but it should still be possible to enable other "modes" as well ("allow" css display property to override the mentioned default behaviour), to allow more flexibility.

As already mentioned, usecases would be websites, which have images as a part of their layout on larger screens (header/teaser images for news articles), but don't want/have to display them on small mobile devices to improve page speeds and reduce data usage on both ends (server doesn't have to send the image, user doesn't have to receive it).

It's totally ok to be considered a nice-to-have. But since this whole <picture>- element was introduced as an option to include lower resolution images on mobile devices natively in HTML (if i remember correctly), it should be considered to be able to "remove" images completely on specific devices. Using workarounds like "empty" data URIs and 1x1 transparent gifs really feels like a hacky way to do it and it shouldn't be the only way to solve this "problem". :wink:

@Wilto Makes an entirely valid point. By all meanings, we should probably respect the picture elements presence as what it is: a picture. That (at least to me) means, that network traffic itself can be no argument on changing the spec for that particular element. So, in my opinion, this discussion should be about browsers loading sources for images that are not displayed (aka: the display: none; problem). By sound logic, an element (like a picture) is either visible/present/displayed or not. That should be the actual entry point for a browser to determine if an image source should be loaded or not.

I partly agree with you, @jeremyzahner. While it is generally true, that images should only be loaded by the browser, if they are visible to the user, there may are some different use cases, where you want to have an image loaded, before it is visible to the user. Example: Fading in images while scrolling the page. In this case, the animation would be played and the images loads, while the animation is playing. And if I understood the intentions for the pictureelement correctly, it was indeed introduced to give people an easy way to distribute lower resolution images to people on mobile devices, so network traffic (and maybe data volume) were an important consideration for this particular element.

If we can tell the browser to load images of different quality/size for different device-sizes, we should also be able to tell the browser to load no image at all for a specific picture tag natively.

[jeremyzahner]

While it is generally true, that images should only be loaded by the browser, if they are visible to the user, there may are some different use cases, where you want to have an image loaded, before it is visible to the user.

@RicoBrase Isn't that what the CSS visibility property is for? 😄

I would agree with @Wilto that we should never hide real content from folks, but then there’s the judgement of when is image content nice-to-have vs. integral to the content.

@aarongustafson You are totally right. I'm just not sure if this use case should be solved by the picture element itself. Since the use case is heavily based upon the context (content vs. nice to have), shouldn't it be solved in the context (by CSS for example)?

[RicoBrase]

@jeremyzahner As long as that is considered for the browser to be "visible to the user" and loads the image initially, than yes, you are right. :wink:

(Just assuming your idea would be actually used by a current browser.)

But that's another topic basicly and should not belong here. :wink:

[jeremyzahner]

@RicoBrase Actually visibility is not my idea, it has been in the spec for a long time.

https://www.w3.org/TR/CSS2/visufx.html#visibility

I'm not quite sure on image loading behaviour by current browser concerning visibility.

But that's another topic basically and should not belong here. 😉

If we might come to the conclusion that some issues should be solved by another spec rather than this one, than that point belongs here (i guess?).

[RicoBrase]

@jeremyzahner

Actually visibility is not my idea, it has been in the spec for a long time.

I know that there is a css property called 'visibility'. "Your idea" related to not let browser load invisible images, not to the css property itself. :wink: Sorry if that was a bit unclear.

While it is generally true, that images should only be loaded by the browser, if they are visible to the user, there may are some different use cases, where you want to have an image loaded, before it is visible to the user.

Isn't that what the CSS visibility property is for?

In your "concept" of a browser not loading invisible images, this should only affect images, that were set to display: none;. visibility: hidden; should not be affected by this (see my example of image-loading animation) or the browser has to detect any possible animations and queue them to be played as soon as the image is done loading.

Therefore i think, this behavior change (allow empty srcset to hide the image or introduce an optional parameter to fulfill this behavior) is absolutely valid. Maybe "both" things should be considered. One option does not exclude the other option. But following the basic idea of this request here would probably require less work on browser side, why i personally would recommend that.

Side note:

You are totally right. I'm just not sure if this use case should be solved by the picture element itself. Since the use case is heavily based upon the context (content vs. nice to have), shouldn't it be solved in the context (by CSS for example)?

We now have the ability already, to choose between different image resolutions (or completely different images at all) natively in HTML. Why should we "outsource" the functionality to load no image at all to CSS now? Your points are valid though, it should be considered (as i said a few lines above) to implement both things, since both ways are absolutely possible. But I personally think it is more intuitively (for me and other programmers/designers) to be able to have the option to load no image inside of a picture-tag in HTML natively.

[aFarkas]

@RicoBrase @jeremyzahner I couldn't fully follow your discussion. But the idea of controlling which image candidate should be loaded or not by the browser via CSS is dead. If it would be a wise idea, some parts of the responsive image markup API (media and especially the sizes attribute) would be redundant and badly designed. But they are not.

You can also read these: https://ericportis.com/posts/2014/separated/ http://calendar.perfplanet.com/2013/big-bad-preloader/

[RicoBrase]

@aFarkas Yeah, as I said:

We now have the ability already, to choose between different image resolutions (or completely different images at all) natively in HTML. Why should we "outsource" the functionality to load no image at all to CSS now?

[BenRacicot]

I like the arguments here. It seems the opinions are split. However I'm still not seeing the complete logic where picture, sourceset or src has no function to ignore an image call. Yet picture even defers its source to the next higher size (as I've brought out in my comment above).

@aarongustafson About content vs context: I agree and believe that the developer is responsible for this. Why is the spec determining what will be displayed/hidden? I also believe that the world-wide bandwidth issue should overrule the content vs context argument. Either way an empty src should load no image.

To add to the above: @aFarkas I agree about CSS. It's a dead end to control network calls. A no-src attribute would be great for this one purpose. Or a display-none attribute? If there existed an html version of display:none; that explicitly does not call the source it is applied to. That seems to make a lot of sense to me...

[yoavweiss]

I think the first step here is deciding that avoiding to display an image in certain viewports is a valid use-case. That use-case can now be served by adding a valid data-URI as the srcset value for the corresponding <source>, but I think it'd be better to enable first class support for that use-case. From reading this thread, it seems like the majority of people here are of similar opinion.

Once we have agreement, I think the simplest way to enable that use-case is to change the processing model algorithm to accommodate that (assuming this is still Web-compatible), or add an opt-in attribute if such a change can no longer be made.

@wilto - what do you say?

[zcorpan]

I would like to check if this use case is served well-enough by adding declarative lazy loading (and for such images, not load at all when they are display:none).

[jeremyzahner]

@RicoBrase @aFarkas Maybe I lost it a little playing the devils advocate here. I totally agree on @yoavweiss' claim that we should agree on the use-case beeing present. I think it is.

[RicoBrase]

@zcorpan That would probably be sufficient, but - IMO - it's way more intuitive to either use an empty src or srcset attribute (or add an additional no-src opt-in attribute), just as @yoavweiss said.

Method 1: If a web designer/developer (or a newbie in this topic) first hears about the picture tag and had to guess, how to prevent the loading of an image on specific devices, he/she would probably first try to use an empty src/srcset attribute. - Well, this could lead to confusion, since empty src attributes on img tags are not allowed although it would be pretty darn good, if it was - but actually there is no particular reason since the src of an image is not changing very often.

Pro: most intuitive Contra: inconsistency in behaviour

Method 2: The next logical thing to is to use an attribute, because that's what they are made for. Change or add behavior to an element (example: type attribute on inputs).

Pro: the 'logical' way, could be implemented for any 'resource type' element (maybe for future elements, who knows?) Contra: Nothing i could think of

Method 3: The CSS way (prevent loading images with css property display: none;). IMO the worst attempt in terms of consistency (load different sized images purely with HTML but to prevent the loading of an image in CSS?) and 'accessability' for programmers/designers. But it would be actually somewhat of the normally expected behaviour of disply: none; for images. This would probably require, that browsers load css rules before sending requests for images/resources, to prevent images from loading, before the css rules are applied (if needed - I have no idea if this is/could be the case).

Pro: Would add expected behaviour to browsers Contra: non-intuitive (IMO), requires probably the most work for browser manufacturers, non-consistent behaviour (regarding the behaviour of the picture tag)

Feel free to add or correct anything, i missed or got wrong. It may help to find the 'best/perfect/ideal' design for this 'problem'. :wink:

@jeremyzahner I agree, too. Actually, that was one of the first things i thought of, when i first heard of the 'picture' tag in HTML 5.1 today.

[aFarkas]

@zcorpan

I would like to check if this use case is served well-enough by adding declarative lazy loading (and for such images, not load at all when they are display:none).

This would also mean, that this pattern shouldn't be used in those cases, where the image is a "hero image" or otherwise initially in view and important for the first layout impression.

plus: If you open the auto sizes pandora box it will be used - always.

[nhoizey]

I got a use case recently, where a site owner would like to show some complex diagrams on viewports large enough, but provide only a download link on smaller ones.

A source with an empty srcset (or no srcset at all maybe) seems to be the best approach IMHO, as it doesn't seem to break anything already implemented or used.

[aarongustafson]

Another great use-case @nhoizey!

@Wilto?

Also, it seems like, as a container, a picture with no supported source might fall back to the inner content, right? That’s the recommended behavior of video & audio.

[tabatkins]

That’s the recommended behavior of video & audio.

Nope, the contents of video and audio are soley for downlevel clients that don't understand the tags at all; they're intended to let you put an object pointing to a SWF equivalent. They don't provide a11y or "fallback" contents at all; the contents are never exposed in supporting clients.

(I just had to look this up for a CSS thing; I was under the same mistaken impression you were last week.)

Also, it seems like, as a container, a picture with no supported source might fall back to the inner content, right?

Nope on this either. :( picture just provides resource-selection instructions to its contained img; it doesn't do anything else. (And doesn't allow anything else; only img and source are valid inside it.) The img falls back to its alt text as normal.

[aarongustafson]

@tabatkins I was getting together a demo on CodePen late last week and realized that was the case. The empty video element when no source is supported kinda sucks. Seems like it might be worth revisiting at some point.