Img: add method to check if image format is supported

[mpetroff]

While all web browsers support the JPEG and PNG image formats, checking for support for additional image formats is difficult from JavaScript (the <picture> element solves this in markup). Currently, detecting support for WebP and AVIF (or JPEG 2000 / JPEG XR) requires creating a single pixel image and seeing if it works [1]. This is less than ideal and will only get worse when / if support for newer image formats, such as JPEG XL and WebP2, is added by browsers.

The solution to this is adding a method that allows for checking if an image format is supported by the browser, so a script that's loading an image can choose the appropriate image file to request (or can send the appropriate Accept header with XHR to let the server decide which file to send). Ideally, this would be similar to the existing HTMLMediaElement.canPlayType() method for video / audio.

[1] https://developers.google.com/speed/webp/faq#in_your_own_javascript

[domenic]

This seems very reasonable to me. Are implementations interested in this?

I suspect that unlike HTMLMediaElement.prototype.canPlayType(), this could return a simple boolean true/false. (But, I might be wrong; implementer weigh-in would be great.)

[domenic]

In https://bugs.chromium.org/p/chromium/issues/detail?id=1197269#c3 it was pointed out that this is mostly handled by the proposal at https://github.com/dalecurtis/image-decoder-api/blob/master/explainer.md .

[mpetroff]

The ImageDecoder.isTypeSupported method supports everything I had in mind, as long as browsers don't partially implement future image formats, as was originally done for WebP, where lossless, alpha, and animation support were added to Chromium later than lossy support. Partial support would necessitate the ability to check for for a specific feature, which I suppose could be implemented with a features= parameter akin to the codecs= parameter used by HTMLMediaElement.prototype.canPlayType().

[jakearchibald]

PNG had partial implementations too, and newer formats are more complicated, so it seems pretty certain that we'll get more partial implementations.

(Also, JPEG defines support for arithmetic compression, but no browser supports it)

[mathiasbynens]

I’m unsure where the draft spec for ImageDecoder.isTypeSupported lives, so apologies if this is the wrong place to post. But I noticed that the flagged Chromium implementation is promise-based, which seems confusingly different from isTypeSupported on MediaSource which synchronously returns a boolean. https://github.com/dalecurtis/image-decoder-api/issues/6 Can we not make ImageDecoder.isTypeSupported synchronously return a boolean as well? Or is it somehow expensive to check whether a given image type is supported?

[mpetroff]

There's an open PR for merging image decoding into the WebCodecs spec, so I assume that's the most recent revision.

[chcunningham]

which I suppose could be implemented with a features= parameter akin to the codecs= parameter used by HTMLMediaElement.prototype.canPlayType().

Let me opine on codecs=. The features of a codec in the codec string are implied by the codecs "profile". For example, a user agent may support the "baseline" H.264 profile but not the "high" profile. Not all codecs have a profile (e.g. most audio codecs), in which cases the implication is that all features of the codec are supported.

The codecs strings are complicated and are typically defined by those involved in specifying the codec features (e.g. https://www.webmproject.org/vp9/mp4/#codecs-parameter-string). The nice thing about strings is it avoids needing to add separate feature descriptions for the multitude of codec features (intractable). But, if such strings do not already exist for images (I'm not familiar enough to know), you'll want to take a great deal of care in minting them, probably involving the appropriate standards body for the given image. Not easy.

[zcorpan]

I suspect that unlike HTMLMediaElement.prototype.canPlayType(), this could return a simple boolean true/false. (But, I might be wrong; implementer weigh-in would be great.)

If browsers will allow video formats in img (https://github.com/whatwg/html/issues/7141) -- which WebKit already does -- it seems to me it should have the same values as video's canPlayType

[annevk]

ImageDecoder.isTypeSupported(type) arguably solves this, but it seems reasonable to also provide an entry point on either img or picture (or on their classes as a static).

Given the discussion in https://github.com/w3c/webcodecs/issues/750 this method will likely have to return a promise that is resolved from a task as well. And perhaps as Simon suggests it should allow for some ambiguity, though it might be weird for it to differ from ImageDecoder semantics.

Colleagues and I would be interested in adding such a method and willing to work on the specification changes needed for it. We were wondering if people had further opinions on the API shape we should take into account.

[othermaciej]

ImageDecoder.isSupported does not obsolete this because there is not (afaik) a requirement for ImageDecoder and img to support the exact same content types. And in particular, it seems likely ImageDecoder would not support vector types like SVG or PDF.

[othermaciej]

Given the discussion in https://github.com/w3c/webcodecs/issues/750 this method will likely have to return a promise that is resolved from a task as well.

Not sure this is true. The reasoning given there applies to WebCodecs but distinguishes it from HTMLMediaElement.canPlayType. Specifically, the difference is passing additional detailed parameters and giving an authoritative answer given the current state of the system, and not ever returning a maybe answer.

[shallawa]

I am not sure how the web page can benefit from knowing the decodable image formats. I think the page will be interested more in knowing whether a certain image is decodable by the browser or not. I think also listing all features for an image format and asking the browser to tell which feature is supported or not will be difficult. On Apple ports we rely on the system frameworks to do the decoding. So we have to keep the list of features in sync with the system frameworks and we have to make sure we reply correctly on down level OSs.

I think the HTMLImageElement.decode() can be used for this purpose. Here is a sample code for how it can be used:

function loadBackgroundImage(element) {
    return new Promise((resolve) => {
        var image = new Image;
        image.src = "image.heic";
        image.decode().then(() => {
            element.style.backgroundImage = 'url(' + image.src + ')';
            resolve();
        }).catch((err) => {
            image.src = "image.avif";
            image.decode().then(() => {
                element.style.backgroundImage = 'url(' + image.src + ')';
                resolve();
            }).catch((err) => {
                element.style.backgroundImage = 'url(image.png)';
                resolve();
            });
        });
    });
}

[annevk]

@othermaciej https://github.com/w3c/webcodecs/issues/750#issuecomment-1832404395 asserts that canPlayType() is a pain as well due to its synchronous nature.