image quality issues

[pixelzoom]

Image quality seems to have taken a turn for the worse after changing rendering strategies in https://github.com/phetsims/function-builder/issues/3.

Here's the previous strategy, which draws HTMLImageElement to Canvas, convert the Canvas back to HTMLImageElement, then uses new scenery.Image(HTMLImageElement). http://www.colorado.edu/physics/phet/dev/html/function-builder/1.0.0-dev.2/function-builder_en.html?screens=4

Here's the current strategy, which draws Canvas to Canvas, and uses new scenery.Image(canvas). http://www.colorado.edu/physics/phet/dev/html/function-builder/1.0.0-dev.3/function-builder_en.html?screens=4

I haven't tested on all platforms, and I'm mainly concerned that the quality on OS X + Chrome has degraded to what I typically see on Firefox.

Here's a comparison of some of the images on OS X + Chrome. You'll probably need to click on these images, expanding them to full size, to see the detail.

1.0.0-dev.2:

1.0.0-dev.3:

1.0.0-dev.2:

1.0.0-dev.3:

[pixelzoom]

I'll need to potentially handle retina issues too. Misc notes:

In the console, should see this on retina devices:

> window.devicePixelRatio
2

Canvas context has backingStorePixelRation, whose names differs by platform. See scenery.Util.backingStorePixelRatio.

[pixelzoom]

Using Image(canvas.toDataURL) instead of Image(canvas) improved quality greatly. Still jaggy on FF, as it typically is.

Some Image options that could be investigated, but have performance implications:

      mipmap: true,
      mipmapBias: 0,

I tried this in CardNode, made FF a bit better, but not great. For now, we'll live with jaggy FF.

[pixelzoom]

Mentioning @amanda-phet and @ariel-phet, so that you're aware of this.

We're going to live with jaggy quality on FF for now, since doing otherwise will adversely affect performance for this sim. Mipmapping is not an option for this sim, and we're going to have to make a tradeoff between responsiveness and image quality (on some platforms).

Closing.

[ariel-phet]

@pixelzoom - please comment on why mipmapping is not an option (I believe you of course, but it would be good to document why)

[ariel-phet]

and feel free to close after a brief explanatory comment

[pixelzoom]

@ariel-phet asked:

please comment on why mipmapping is not an option

Changing a function (or composed function) needs to immediately change multiple images: the images that the user(s) are dragging, the image(s) that are visible in the output carousel (which may be in the process of being scrolled.) And in the case of a composed function (2-3 functions chained together) this is a multi-step process, where the intermediate images need to be retained, for viewing via the spy glass feature. So image processing needs to be highly optimized in order to keep the simulation responsive to user input.

Mipmapping is inherently expensive, a tradeoff between quality and speed. Mipmap support is available in 2 places:

(1) When an image is loaded, by using the mipmap.js plugin. This creates an array of images at various scales that can be passed to scenery.Image, which automatically selects the best image to render for the desired scale. Since this sim must process the image data, using mipmap at this stage greatly increase the amount of data that needs to be processed - we would need to process multiple images every time a function is applied, and create multiple canvases with different sizes. We simply cannot afford to do that. So we're using the image.js plugin to read image files (which returns a single image), writing it to a canvas, then doing image processing on the image data in the canvas.

(2) As options to scenery.Image, see options: mipmap, mipmapBias, mipmapInitialLevel, mipmapMaxLevel. These options allow Image to create a mipmap from a non-mipmapped image. Setting mipmap:true triggers creation of mipmaps of medium quality and (as the documentation says) "this mipmap generation is slow and CPU-intensive". I experimented with this in the "Test" screen, and it was much much slower. So this is something that we cannot afford to do.

Questions?

[ariel-phet]

Nope...thanks for the explanation, useful for me to understand the constraints when discussing potential future designs and such!