Support for clipping (clipWith?)

[kleisauke]

Hi there!

It would be awesome if this library supports clipping out-of-the-box. Currently I'm doing this with these steps:

• Convert the image to PNG (using sharp)
• Encode the image to a base64 encoded image (data:image/png;base64)
• Put the base64 encoded image inside in the SVG xlink:href= attribute (inside the <image> element which has for example the clip-path="url(#clipCircle)" attribute)
• Change the width and height of the SVG to the original PNG dimensions
• Save the SVG (example here)
• Load the SVG into sharp and convert it back to PNG.

To simplify these steps it seems to me a good idea to come with a new API feature (this is just a draft):

clipWith([options])

Clips an image to a certain path. The parts of the shape inside the path are visible, and the parts outside are invisible.

options, if present, is an Object, Buffer or String with the following optional attributes:

• shape is a String or an attribute of the sharp.shape Object e.g. sharp.shape.circle or sharp.shape.ellipse
• Buffer containing SVG image data
• String containing the path to an SVG image

For example I want to crop this image to a circle. Then I just do this:

sharp('500x500.png')
  .clipWith({ gravity: sharp.shape.circle })
  .toFile('500x500-circle.png', function(err) {
    // 500x500-circle.png is a 500 pixels wide and 500 pixels high image
    // containing a clipped to circle version of 500x500.png
  });

Which outputs this image.

I don't know how much demand there is for this feature, but this will help for me to write more efficient and less hackish code. (The way I'm doing it right now is way too complicated :expressionless:)

URL's which might be helpful in order to accomplish this feature: draw: VIPS Reference Manual RSVG Libary Reference Manual Clippy — CSS clip-path maker A few SVG files to test the SVG <clipPath> element

[lovell]

Hello, assuming the output dimensions are known, you might be able to use the existing overlayWith operation to composite an image (PNG, WebP, GIF or SVG) containing a transparent circle over the top of your input images.

[kleisauke]

The output dimensions are known, but may vary. I've tried with existing overlayWith but that doesn't work for me.

Let's assume that the dimensions are always 500x500 and that I use this SVG overlay:

<svg xmlns="http://www.w3.org/2000/svg" width="500" height="500" viewBox="0 0 100 100">
    <circle r="50" cx="50" cy="50" fill-opacity="0.0"/>
</svg>

For the input I'm using this PNG:

Test 1:

var path = require('path');
var sharp = require('sharp');

sharp(path.resolve(__dirname, '500x500.png'))
  .overlayWith(path.resolve(__dirname, 'circle.svg'))
  .toFile('500x500-circle.png');

Then the file 500x500-circle.png is exactly the same as our input file.

Test 2: Let's change the SVG overlay to this:

<svg xmlns="http://www.w3.org/2000/svg" width="500" height="500" viewBox="0 0 100 100">
    <circle r="49" cx="50" cy="50" stroke="black" stroke-width="1" fill="white" fill-opacity="0.0" />
</svg>

And running the same test as test 1, then the output is this: (It's not cropped into the circle)

Test 3: Let's change the SVG overlay to this:

<svg xmlns="http://www.w3.org/2000/svg" width="500" height="500" viewBox="0 0 100 100">
    <path d="M0,50v50h50C22.4,100,0,77.6,0,50z" fill="white"/>
    <path d="M50,0H0v50C0,22.4,22.4,0,50,0z" fill="white"/>
    <path d="M50,100h50V50C100,77.6,77.6,100,50,100z" fill="white"/>
    <path d="M50,0c27.6,0,50,22.4,50,50V0H50z" fill="white"/>
</svg>

And running the same test as test 1, then the output is this: (Cropped into the circle but with white borders; not transparent)

Tested on Windows 10 x64, Node.js v5.9.1 with the latest Sharp (v0.15.0)

[lovell]

Thank you for the clear examples. The only way to have transparency with the current API is if the image to be overlaid contains the transparency, which sadly won't be the case in this scenario.

How about a transparency operation (transparencyFrom?), which would accept a single channel image (or use the alpha channel of a multi-channel image) and make it the transparency layer of the output image?

[kleisauke]

I'm not sure if the transparencyFrom operation is the right thing to solve this. How about alpha compositing using the libvips morphology. (Specifically vips_orimage())

Draft operation (inspired by this):

mask(image, [mask_with_alpha])

Apply a given image source as alpha mask to the current image to change current opacity. Mask will be resized to the current image size. By default a greyscale version of the mask is converted to alpha values, but you can set mask_with_alpha to apply the actual alpha channel. Any transparency values of the current image will be maintained.

image is one of the following, and must be the same size (or else it will resized to the current image size):

• Buffer containing PNG, WebP, GIF or SVG image data, or
• String containing the path to an image file, with most major transparency formats supported.

mask_with_alpha is a Boolean where true applies the actual alpha channel as mask to the current image instead of the color values, defaulting to false

Then I'll just change the SVG to this:

<svg xmlns="http://www.w3.org/2000/svg" width="500" height="500" viewBox="0 0 100 100">
    <rect width="100%" height="100%" fill="black"/>
    <circle r="50" cx="50" cy="50" fill="white"/>
</svg>

[lovell]

Your eloquently-described mask operation is pretty much what I was thinking with transparency but you did a far better job of explaining it, thank you!

Instead of having black vs white logic, perhaps we could directly "draw" the transparency. This means we could simplify your example further so you'd need only the circle element:

<svg xmlns="http://www.w3.org/2000/svg" width="500" height="500" viewBox="0 0 100 100">
    <circle r="50" cx="50" cy="50"/>
</svg>

The circle would represent the pixels we want to "keep". The transparency value of the pixels in the "non-circle" around the edges would become the transparency values of the pixels of the image underneath.

"Mask will be resized to the current image size"

I'd prefer to avoid adding resizing logic in here as there are many options involved (kernel? interpolator? crop? embed? aspect ratio?).

I see the API having more in common with the existing compositing feature of the withOverlay operation, so for example you could specify an optional gravity.

This means any resizing, if required, would need to be done up-front. For SVG this should be controllable via the width/height/viewBox attributes.

[kleisauke]

Directly draw the transparency would be great!

I agree with you about avoiding the resizing logic. Also I think there are too many options involved in this process which you described.

This feature has indeed a lot of common with the existing overlayWith operation. How about a new option (called cutter) in the overlayWith operation?

See this draft:

overlayWith(image, [options])

Overlay (composite) a image containing an alpha channel over the processed (resized, extracted etc.) image.

image is one of the following, and must be the same size or smaller than the processed image:

• Buffer containing PNG, WebP, GIF or SVG image data, or
• String containing the path to an image file, with most major transparency formats supported.

options, if present, is an Object with the following optional attributes:

• gravity is a String or an attribute of the sharp.gravity Object e.g. sharp.gravity.north at which to place the overlay, defaulting to center/centre.
• cutter is a boolean where true trim pixels according to the transparency levels of a given overlay image. Whenever the overlay image is opaque, the original is shown, and wherever the overlay is transparent, the result will be transparent as well. Defaulting to false.

[lovell]

Great, this keeps things simple. Perhaps cutout is a little more descriptive as an attribute name.

Thank you for this feature suggestion Kleis!

[lovell]

In terms of implementation (I spotted your question on the libvips' repo), if we're willing to throw away any existing alpha channel on the underlying image, then this might be as simple as splitting the alpha channel from the overlay and joining it to the image underneath.

This means R1G1B1 overlaid with R2G2B2A2 would become R1G1B1A2 (and R1G1B1A1 overlaid with R2G2B2A2 would become R1G1B1A2).

The two images are currently premultiplied before and the resultant image unpremultiplied after. In the case of a band split/join, that step could probably be removed to improve performance, although it shouldn't do any harm beyond a chance of a small rounding error if still used.

[kleisauke]

A pull request has been made for this at #448.

[odbol]

This is great. Exactly what I needed to add rounded corners to an image. Will it be merged soon? How do I use it?

[lovell]

Docs/changelog updated via commit c3ad4fb. Thanks again @kleisauke for all your work on this, which will be in the forthcoming v0.15.1 release.

[lovell]

v0.15.1 now available, thanks again.

[Zerocool27]

I cannot see this function anymore :(

[lovell]

@Zerocool27 Use the dest-in blend mode of the composite operation.