Open Salmondx opened 1 year ago
Hi Gleb,
Just wanted to say thanks for the PR, which I pulled into my own fork and successfully tested.
I also added a counterpart feature to export the whole canvas as raw data asynchronously. It's sort-of like context.getImageData()
but for the whole canvas and using worker pool threads (like toBuffer()/etc do). The result can be returned as Buffer
or ImageData
types or written to file as usual.
Also added a feature to specify a crop rectangle when exporting in any raster format or raw.
I'm using all this in a workflow that reads in various images formats with sharp, resizes and caches them in memory. Then later we draw them to a Skia Canvas (possibly with other elements, transforms, etc), export the canvas back to sharp
which does final PNG compression (and possibly tiling into multiple parts) and then the resulting data is sent to a network client. We use sharp
for compression because it (or rather libvips
) is way, way better at it than what we can currently get out of skia-canvas (which of course has no compression options for PNG at all).
The drawing/compression/sending part can happen very often with multiple images possibly being requested in the same millisecond with refreshes at 10Hz or more being common. So, performance matters. Switching to raw processing, skipping unnecessary encoding steps, has been benchmarked as ~400% overall improvement in speed, w/out any extra CPU load or memory use. Nice win.
Anyway, it's all published, and documented in the changelog and readme if anyone cares to take a look. There are 2 branches which contain the core changes, though one builds upon the other.
I should mention that this is my first ever Rust project, so quite likely there are some prettier or more efficient ways of doing some things. The syntax and some concepts have a bit of a learning curve!
Cheers, -Max
Oh I forgot to add one caveat I found when loading raw pixel data to SkImage. Skia doesn't support decoding 3-channel/24-bit color pixels like RGB. It can only handle 1, 2, 4 or 8 byte formats (reference). So when trying to "raw-load" some PNG I had which was RGB only, Skia returns nothing.
ColorType.bytesPerPixel()
returns 4
for RGB888x
image type (whereas, for example, sharp
properly shows such images as having only 3 channels).
Luckily sharp
has an ensureAlpha()
function which will add an alpha channel if it's missing to round the pixel data out to 32 bits in these cases. But it's something to watch out for.
Description
Based on #113
This PR adds a new functionality to
loadImage()
andImage
to load images from already decoded pixel buffers. Currently there is no way to do that andskia-canvas
only supports loading images from encoded formats and does encoding on its own using Skia methods.There are many cases when loading raw pixel buffers are beneficial. For example, when reading images from web cams, reading video frames from ffmpeg or while using custom image decoders that Skia doesn't support.
Details
In this implementation I added a new optional parameter called
ImageOptions
that currently supports only one optional property calledraw
of typeImageData
. API consumer can provide this object toloadImage()
function and toImage
constructor. IfcolorType
and passedimage resolution
doesn't match pixel buffer data, a regularError
will be thrown using existing callbacks.Example with
loadImage()
:Example with
Image
constructor:In Rust code I created a new function called
load_pixel_data
that is responsible for creatingSkImage
. It doesn't align perfectly with existing code that uses getters/setters and I didn't find a suitable way of how to make it look better. May be we can removeset_data
setter and make it a function as well but I didn't want to do that without consulting with you first.Also currently only the most common pixel formats are supported (such as
rgba
,rgb
,bgra
andargb
) and I'm not sure if we need to expose other less popular formats that Skia supports.Tests
I added several tests that test different loading scenarios and also tested internally using different popular libraries that allow raw pixel buffer extraction (for example
sharp
andffmpeg
). Based on my simple benchmarks I didn't find anything unusual, CPU and RAM usage was normal.