search

python-pillow / Pillow

Python Imaging Library (Fork)
https://python-pillow.org
Other
12.21k stars 2.22k forks source link

Paste issues #1911

Closed simonschofield closed 8 years ago

simonschofield commented 8 years ago

What did you do?

I am using pillow image paste to construct large collage images from many small ones. The large target image is initially set to have an overall alpha of 0, so is like an empty transparent Photoshop layer if you like. I am pasting smaller images with various alphas onto this You can see such images at www.simonschofield.net.

What did you expect to happen?

I expect that the paste function mimics the Duff Porter OVER composite algorithm

What actually happened?

I have noticed two things wrong with paste. 1/ If you paste onto a fully transparent area of the target canvas image it seems to blend with a mid-grey creating dark halos around light objects. This is not in accordance with Duff Porter: it should set the colour to that of the incoming pasted pixel, and set the alpha to the same as the incoming pixel. 2/ Multiple applications of Alphas of pasted pixels to the same point in the large canvas image should should always contribute (i.e. always get greater, stopping at 255). I have noticed that pasted images with low alpha seem to lessen the alpha of the existing pixels in the canvas image, making those areas more transparent.

Also, is paste the only way of compositing a small image into a larger one?

What versions of Pillow and Python are you using?

Pillow is about v3 Python 3.3.3 64bit

wiredfool commented 8 years ago

You might want to look at Image.alpha_composite: http://pillow.readthedocs.io/en/3.2.x/reference/Image.html?highlight=alpha_composite#PIL.Image.alpha_composite

simonschofield commented 8 years ago

Thanks, sounds promising. Will do. Is this a new feature, I don't remember seeing it in my in my version?

simonschofield commented 8 years ago

Ahh but it says the source must have the same size as the target image, I want to paste smaller images into a big image.

simonschofield commented 8 years ago

Ignore last email. I was looking at blend by mistake

simonschofield commented 8 years ago

No, alpha composite need two images of the same size. I want to composite image A at point P within image B

simonschofield commented 8 years ago

The documentation admits my second point

"Where the mask is 0, the current value is preserved. Intermediate values will mix the two images together, including their alpha channels if they have them."

This is not useful in a standard paste operation and should be altered to accumulate the alpha or at least have the choice with a flag

wiredfool commented 8 years ago

Does alpha composite do the right thing if it could compose a small image over a large one?

syranide commented 7 years ago

@wiredfool I've encountered this as well, the current behavior seems useless and not what one would expect. The only way around it I've found is really hacky. Are you amenable to changing this behavior? (EDIT: Or adding a companion function that exhibits this behavior)

Alpha composite is not applicable to my use-case as I'm trying to render an image mask with different colors (i.e. setting im to a color a value rather than an image).

wiredfool commented 7 years ago

In general -- we don't tend to want to break existing use of the function.

Practically, a parameter to paste, like disposition, may be appropriate, or alpha_composite might be the right approach, if extended to take a region of interest.

syranide commented 7 years ago

@wiredfool alpha_composite isn't really well suited for many small operations though as it's not inplace. So by that logic extending paste with a new argument would make most sense? Or perhaps inplace + box for alpha_composite?

wiredfool commented 7 years ago

Is alpha_composite the operation that you're looking for? (leaving aside efficiency)

syranide commented 7 years ago

@wiredfool Sorry, got a bit distracted there. No it's not, I'm specifically using paste with a solid color and a mask, so that I can dynamically change the color of what I'm rendering. alpha_composite does not support that, and as far as I can tell alpha_composite is pretty much the same thing as paste with just an image (which works fine, it's paste with a mask that doesn't).

In general -- we don't tend to want to break existing use of the function.

Just so that it is said. Pasting an transparent image onto a transparent image works fine, pasting a mask onto an opaque image works fine, pasting a mask onto a transparent image mixes colors from the destination regardless of destination alpha. This seems backwards and not at all useful to anyone, IMHO it really seems like a bug. But I get it if you want to keep it that way.

PS. Even paste isn't perfect for my use-case, I'm pasting pieces from an alpha spritesheet so I first have to crop out the sprite and then paste that onto the final image. But it works well enough.

wiredfool commented 7 years ago

So, I'm not really sure that I understand what you're aiming for. Here's my understanding:

In stage 3, paste with a hard mask will replace the existing pixels with the new one. paste with a partial mask will blend the pixels. alpha_composite will use the alpha mask from the sprite to blend it in. (e.g., monochrome sprite on background color, so a duotone)

E.g.:

(vpy27)erics@builder-1204-x64:~/test$ python
Python 2.7.3 (default, Aug  1 2012, 05:14:39) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from PIL import Image
>>> im = Image.open('Tests/images/hopper.png').convert('L')
>>> sprite = Image.new('RGBA', im.size, 'red')
>>> sprite.putalpha(im)
>>> sprite.save('issue_1911_sprite.png')

issue_1911_sprite

>>> background = Image.new('RGBA', (200,200), 'blue')

>>> mask = Image.new('L', sprite.size, 'white')
>>> mask.paste('black', (0,0,64,64))
>>> mask.paste((128), (65,65,128,128))
>>> mask.save('issue_1911_mask.png')

issue_1911_mask

>>> paste = background.copy()
>>> paste.paste(sprite, box=(32,32), mask=mask)
>>> paste.save('issue_1911_pasted.png')

issue_1911_pasted

>>> alpha = Image.alpha_composite(background.resize(sprite.size), sprite)
>>> alpha.save('issue_1911_alpha.png')

issue_1911_alpha

So does one of these operations cover what you want as output? If so, which one, and if not, can you explain with some sample code?

syranide commented 7 years ago

@wiredfool Sorry for the delay, been out of country. Sorry I thought I was being more clear, should've posted my testcode from the start. Yes that is exactly what I'm expecting, but that's not what pillow gives me when pasting a mask on a non-opaque background.

See:

from PIL import Image

im = Image.open('img.png')

im1 = Image.new('RGBA', (im.width, im.height), (0, 255, 0, 0))
im1.paste(im)
im1.save('a0.png')

im1 = Image.new('RGBA', (im.width, im.height), (0, 0, 255, 0))
im1.paste(im)
im1.save('a1.png')

im1 = Image.new('RGBA', (im.width, im.height), (255, 0, 0, 0))
im1.paste((0, 255, 0), mask=im)
im1.save('b0.png')

im1 = Image.new('RGBA', (im.width, im.height), (0, 0, 255, 0))
im1.paste((0, 255, 0), mask=im)
im1.save('b1.png')

a0 a1

I would expect a0.png and a1.png above to be the red transparent dude, they are.

b0 b1

I would expect b0.png and b1.png above to be a green transparent dude, they are not, b0.png has a red tinge and b1.png a blue tinge, sampled from the totally transparent background. Because even though it's totally transparent the color channels are still mixed in.

syranide commented 7 years ago

Now I see that paste doesn't alpha composite at all, which I thought it did. But when using mask it kind of does, but badly as above. I'm trying to render text (custom font system), so I need to alpha composite individual letters from a mask sprite sheet (which can also be colored then). So paste with color, bbox and mask given that it would alpha composite correctly is exactly what I need. But plain paste does not alpha composite and so does not work. alpha_composite only takes takes two equally sized images and so is clunky/slow, it requires me to first use paste to render the character in the desired color to a separate buffer (final image sized) and then alpha composite that into the final image, thus resulting in a new buffer, for each letter.

wiredfool commented 7 years ago

(FYI, the 'dude' is Admiral Grace Hopper, the writer of the first compiler, namer of the computer bug, distributor of nanoseconds.)

Masks are not alpha channels. Similar but different ideas.

A mask is a selection, possibly partial, that will select an area to paste. It's similar to a selection mask in photoshop, though there might be implementation details.

Alpha is a transparency function, and alpha composite uses that transparency as a mask to overlay the second image on the first.

As for the last two images, they are poorly specified in 2 ways:

If I were to do this with the current tools, I'd do this:

syranide commented 7 years ago

If I were to do this with the current tools, I'd do this:

@wiredfool Yeah that makes a lot of sense actually. However, we support kerning so the bounding boxes overlap, which means I require alpha compositing for each letter. So it seems paste is not usable for this and alpha_composite is too heavy-handed (both because it returns a new buffer and requires an equally sized input buffer).

I.e. my use-case requires an alpha compositing paste (if it could have a source bbox as well that would be even better). I'm a little surprised this doesn't already exist considering it is generally a core feature of most libraries.

wiredfool commented 7 years ago

So, to finally circle around to answer the original question -- alpha composite is the proper image operation, and if it were extended using a bounding box for placement it would solve your problem.

syranide commented 7 years ago

@wiredfool alpha_composite is the right image operation yes and the one I'm currently using. Let's put it like this, for my use-case I see the following possible enhancements of alpha_composite:

  1. Support "target point/bounding box". So I no longer have to pad each letter before rendering to final buffer.
  2. Support "inplace". So that each character no longer generates a new final-sized buffer (slow).
  3. Support "source bounding box". So that I do not need to crop letters out before-hand (although I still might for coloring purposes).
wiredfool commented 7 years ago

Thinking about this a bit more, it's possible that the procedure here: https://github.com/python-pillow/Pillow/issues/1911#issuecomment-308869854 would work.

If, instead of pasting in a flat color in the bounding box, you added a mask of the letter as well, filtered through a point function where f(0) = 0, and f(x>0) = 255. That would solve the bounding box overlap problem, at least for any letters that didn't physically intersect.

I'm looking at the alpha composite issue as well, I've got a trial implementation in python here: https://github.com/wiredfool/Pillow/tree/issue_1911 . It implements a method on PIL.Image.Image: im.alpha_composite(over_img, box), where the source/background image is modified in place. (sort of, but at an API level, that's what's going on. It's going to require tweaking on the C level to actually do it in place. But if this is an API we can live with, then that's a start ).

syranide commented 7 years ago

@wiredfool Looks good, but I would add a source-point parameter as well considering "box" already crops the source, it seems weird not to allow control over the positioning too then.

wiredfool commented 7 years ago

Ok, I've added the source. I'm not 100% on how the size is specified as an optional part of the 4 tuple. There has to be only one size, I'm not into adding a scale factor in this, as ultimately it's going to wind up in C and I'd like for it to be as straightforward as possible at that layer.

syranide commented 7 years ago

@wiredfool :+1: Yeah scaling doesn't make sense. Just a nitpick, personally I think it makes more sense logically to specify size in the source-argument rather than target-argument, but end result is the same.