Added functionality to load images from decoded pixel buffers

[Salmondx]

Description

Based on #113

This PR adds a new functionality to loadImage() and Image to load images from already decoded pixel buffers. Currently there is no way to do that and skia-canvas only supports loading images from encoded formats and does encoding on its own using Skia methods.

There are many cases when loading raw pixel buffers are beneficial. For example, when reading images from web cams, reading video frames from ffmpeg or while using custom image decoders that Skia doesn't support.

Details

In this implementation I added a new optional parameter called ImageOptions that currently supports only one optional property called raw of type ImageData. API consumer can provide this object to loadImage() function and to Image constructor. If colorType and passed image resolution doesn't match pixel buffer data, a regular Error will be thrown using existing callbacks.

Example with loadImage():

let img = await loadImage(pixelBuffer, {
  raw: {
    width: 1920,
    height: 1080,
    colorType: 'rgba'
  }
})
ctx.drawImage(img, 100, 100)

Example with Image constructor:

let img = new Image({
  raw: {
    width: 1920,
    height: 1080,
    colorType: 'rgba'
  }
})
img.src = pixelBuffer
ctx.drawImage(img, 100, 100)

In Rust code I created a new function called load_pixel_data that is responsible for creating SkImage. It doesn't align perfectly with existing code that uses getters/setters and I didn't find a suitable way of how to make it look better. May be we can remove set_data setter and make it a function as well but I didn't want to do that without consulting with you first.

Also currently only the most common pixel formats are supported (such as rgba, rgb, bgra and argb) and I'm not sure if we need to expose other less popular formats that Skia supports.

Tests

I added several tests that test different loading scenarios and also tested internally using different popular libraries that allow raw pixel buffer extraction (for example sharp and ffmpeg). Based on my simple benchmarks I didn't find anything unusual, CPU and RAM usage was normal.

[mpaperno]

Hi Gleb,

Just wanted to say thanks for the PR, which I pulled into my own fork and successfully tested.

I also added a counterpart feature to export the whole canvas as raw data asynchronously. It's sort-of like context.getImageData() but for the whole canvas and using worker pool threads (like toBuffer()/etc do). The result can be returned as Buffer or ImageData types or written to file as usual.

Also added a feature to specify a crop rectangle when exporting in any raster format or raw.

I'm using all this in a workflow that reads in various images formats with sharp, resizes and caches them in memory. Then later we draw them to a Skia Canvas (possibly with other elements, transforms, etc), export the canvas back to sharp which does final PNG compression (and possibly tiling into multiple parts) and then the resulting data is sent to a network client. We use sharp for compression because it (or rather libvips) is way, way better at it than what we can currently get out of skia-canvas (which of course has no compression options for PNG at all).

The drawing/compression/sending part can happen very often with multiple images possibly being requested in the same millisecond with refreshes at 10Hz or more being common. So, performance matters. Switching to raw processing, skipping unnecessary encoding steps, has been benchmarked as ~400% overall improvement in speed, w/out any extra CPU load or memory use. Nice win.

Anyway, it's all published, and documented in the changelog and readme if anyone cares to take a look. There are 2 branches which contain the core changes, though one builds upon the other.

I should mention that this is my first ever Rust project, so quite likely there are some prettier or more efficient ways of doing some things. The syntax and some concepts have a bit of a learning curve!

Cheers, -Max

[mpaperno]

Oh I forgot to add one caveat I found when loading raw pixel data to SkImage. Skia doesn't support decoding 3-channel/24-bit color pixels like RGB. It can only handle 1, 2, 4 or 8 byte formats (reference). So when trying to "raw-load" some PNG I had which was RGB only, Skia returns nothing.

ColorType.bytesPerPixel() returns 4 for RGB888x image type (whereas, for example, sharp properly shows such images as having only 3 channels).

Luckily sharp has an ensureAlpha() function which will add an alpha channel if it's missing to round the pixel data out to 32 bits in these cases. But it's something to watch out for.