Add API for the feature detection of ImageBitmap creation options

[junov]

Currently, createImageBitmap takes an ImageBitmapOptions argument, which is a dictionary. As new options get added to the specification, we will have issues with implementations that do not yet support the new options. Therefore, we should make it easy to feature-detect which options are supported. Ideally it should be possible to make these queries without creating an ImageBItmap object.

[junov]

I can think of several forms this could take. Ideas: 1) A static method getSupportedOptions() on the ImageBitmap interface that returns an array of strings representing the names of supported options 2) A static method getSupportedOptions() on the ImageBitmap interface that returns an associative array (dictionary) 3) A static method isOptionSupported(DOMString optionName)

Anyone know of precedents in other parts of the spec for solving this type of problem?

[domenic]

In general feature detection of options is somewhat of an unsolved problem, see https://github.com/heycam/webidl/issues/107.

[junov]

That is an interesting thread. Assuming there will be no general pattern for the foreseeable future, do you agree we should solve this via API? In the WebGL spec, they solved this for WebGLRenderingContext by adding a method called getContextAttributes() (it is non-static). This method returns an associative array that represents the "actual context attributes", as opposed to the requested ones.

This approach used in WebGL is more flexible than the ideas in my previous comment because it allows enums values to be feature detecteg. What I mean is that just exposing that a given enum attribute is supported does not tell you which values for that attribute are supported by the user agent. The downside is that you only find out that a feature is not supported after trying to use it, which means you have to create dummy objects just for feature detection. :-(

[domenic]

I guess I'd like to be convinced that the dummy objects are really that bad before trying to solve this with a new API.

[kenrussell]

In this particular case I would advocate for a static method like either ImageBitmap.getSupportedOptions() or ImageBitmap.isOptionSupported(). The reason is that developers need to know before deciding to use ImageBitmap at all whether it's going to work as they expect. For WebGL developers, if the imageOrientation attribute is unsupported, they need to fall back to using HTMLImageElement to load their texture resources. Creating a dummy ImageBitmap is for this purpose will make web applications' startup sequence more complex.

[fserb]

Do we see use cases where there would be a lot of trial and error on ImageBitmap creation? Like: try this setting, then try this other one, etc, etc?

Creating a single ImageBitmap that fails because what you want is not supported, is not that complex. But if use cases are closer to webGL (i.e., you may want something but if it doesn't exist you are still fine) then I can see the need for a new API.

[kenrussell]

Most likely the developer would just test to see whether ImageBitmaps' imageOrientation creation attribute is supported, and if not, fall back to using HTMLImageElement rather than ImageBitmap.

It's quite painful to test whether these features are fully supported today. Doing so requires the "rest" of the application to be put inside a "then" block after the creation of an ImageBitmap attempting to use the feature.

I wish that we'd been able to enforce that at least these core attributes -- imageOrientation, premultiplyAlpha, colorSpaceConversion -- were supported in the (WebGL) conformance tests. Unfortunately, today, some browsers don't support them, leading to a lot of developer pain. The popular library Three.js is likely going to have to implement user agent detection to figure out when to try to use ImageBitmaps in preference of HTMLImageElement for higher performance.

Adding this query API and making it easy to use (i.e., make it static, not per-ImageBitmap) is the most forward-looking solution.

[domenic]

Do we believe that this feature-detection API will be implemented faster than those core attributes will be, in all four browsers? Otherwise people are going to have to use user-agent detection anyway.

[kenrussell]

@domenic if Chrome implements the query -- which could be done very quickly -- then feature detection can be made a lot easier. If the query doesn't exist, don't try to use ImageBitmap -- otherwise, do the query, and if the attributes aren't supported, do the fallback.

[domenic]

I'm not sure if we should be encouraging APIs that make you only use new features in Chrome. It seems strictly better for app developers to use a feature detection mechanism that works in all browsers, even if that requires some extra .then(() => {})ing.

[kenrussell]

I'm not sure what you mean. The intent would be for all browsers to implement the new API. The feature detection would look like:

if (ImageBitmap.getSupportedOptions && ImageBitmap.getSupportedOptions()["imageOrientation"]) { // Use the ImageBitmap path } else { // Use the HTMLImageElement path }

Doing this with a "then" requires the same kind of detection, only it's more complicated because it requires waiting for a promise to be resolved.

[domenic]

Yes, but doing it with then will work in browsers that don't implement the new API, instead of forcing them onto the fallback path.

[spite]

I've used CreateImageBitmap and web workers to load and decode images in a couple of projects recently -with great success when everything works, I must say!-, and I'm looking to add it as a feature for three.js ImageLoader. An easier feature detection would be very helpful indeed!

Right now, for the regular createImageBitmap usage:

• it's working in Chrome
• it's not available in Safari or Edge
• I need to sniff Firefox because of this bug if I don't know beforehand the size of the image -which is usually the case!

Another cool application of this API is using a web worker + fetch + createImageBitmap + (sub)texImage2D path to upload images to WebGL textures -freeing the main thread while downloading and decoding the image. Right now there's a noticeable performance drop in Firefox (both desktop and mobile) using this option, so the UA is sniffed and skipped if it's Firefox.

When using subTexImage2D it's important to know about imageOrientation. That's an instance where detection would be very useful.

Maybe if the detection mechanism is first implemented in Chrome, and Firefox can follow when they fix their implementation of createImageBitmap, it will be a good place for the rest of browsers to pick up?

[junov]

@spite: thanks for the feedback.

@kenrussell: The example you give for sniffing "imageOrientation" support is all fine, but it is skipping over my earlier argument about open-ended enums. Here is a concrete example we are likely to encounter in the near future. We will be adding adding color space and bit depth options for canvases (and ImageBitmaps). The current proposal has "8-8-8-8", "10-10-10-2", "12-12-12-12", and float16. Only "8-8-8-8" will be mandatory. To know whether a browser supports a given format, it will not be enough to sniff ImageBitmap.getSupportedOptions()["pixelFormat"])

The argument that devs "need to know before deciding to use ImageBitmap" is not aproblem IMO. Pretty much all polyfills and frameworks do all the feature detection once at startup. With an API that exposes "actual options" after the fact, you'd just do this:

var is12bitImageBitmapSupported = false;
var dummyImage = new Image();
dummyImage.src = "data:...";
dummyImage.onload = function() {
  createImageBitmap(dummyImage, {pixelFormat: "12-12-12-12"}).then(
    image => {
      is12bitImageBitmapSupported = image.getCreationOptions().pixelFormat == "12-12-12-12";
    });
}

[annevk]

(Aside: the color depth syntax looks rather sad.)

[junov]

@annevk: If you have better ideas for color depth syntax, please join our party: https://discourse.wicg.io/t/canvas-color-spaces-wide-gamut-and-high-bit-depth-rendering/1505

[annevk]

I filed issues at https://github.com/WICG/canvas-color-space.

[RByers]

We keep bumping up against this problem in various APIs. Is it really that much harder to agree on some general-purpose compiler-automated pattern in WebIDL that we can start opting new dictionary types into, instead of continuing to add one-off APIs to the web for this?

I still really want to go back and add feature detection to EventListenerOptions but there's really nowhere good to hang a 'getSupportedOptions` method in that case.