Drawing with a colour with alpha < 255 doesn't work as expected

[humdingerb]

Choose a foreground colour like pure blue 0,0,255 and reduce the alpha value to 50%. Drawing anything should end up with the background being slightly visible through the half-transparent ink. But it's only a fully opaque lighter blue.

[humdingerb]

I should note, it does work when you do the drawing on a layer above the background image. Still, it's not really perfect either. For example, if you use the air brush tool (any tool really, but this has immediate connection to reality), if you spray a second time over an area, the half-tranparent colour should add to the one already there to make it more opaque. With ArtPaint, it doesn't.

[PeteCA]

On Fri, Apr 22, 2022 at 10:36:23AM -0700, humdinger wrote:

Choose a foreground colour like pure blue 0,0,255 and reduce the alpha value to 50%. Drawing anything should end up with the background being slightly visible through the half-transparent ink. But it's only a fully opaque lighter blue.

I think this is essentially correct. When you paint into a layer you are always completely replacing all the pixels that were there before, at least for the pen, rectangle and ellipse. Using a brush with faded edges actually seems to properly blend anything that was there before. The specified transparency is set into the pixel of the layer, so that it is transparent to other layers.

[humdingerb]

At least that's not how I expect it to work. I expect to see the existing image through my layer of half-transparent paint, no matter if there are different layers involved. I think most graphics apps work like this.

[dsizzle]

I have spent a bunch of time dealing with alpha trying to understand some of these issues. My current hypothesis is that the compositing calculations seem to be correct. However I think the way some of the brushes are rendered is odd.

In other words, I believe if you draw a solid rectangle at 50% alpha, and place it over another solid rectangle at 50% alpha, the overlap will be 100% opaque and the correct color. But, if you use the airbrush or even the normal brush with fuzzy edges, the transparent parts don't look correct where they overlap. I'm not by my computer right now but I am pretty sure this is what I've concluded after pulling out a lot of my hair. 😆

[dsizzle]

Believe me, I am already considering redoing the entire layering/rendering stack...lol

[humdingerb]

In other words, I believe if you draw a solid rectangle at 50% alpha, and place it over another solid rectangle at 50% alpha, the overlap will be 100% opaque and the correct color.

Not sure this right. At least from other graphics apps (mainly WonderBrush), it seems that the opacity should be multiplied. Like a 50% rectangle on another 50% rectangle would have 75% of the solid colour in the intersection. Add another 50% rect, and you'd get 87,5%. Another for 92,75% and so on.

Maybe the WonderBrush code has some hints. Stippi explained at one BeGeistert how he improved esp. this kind of blending in WonderBrush v3 compared to v2. I didn't understand it back then either, I'm just not a real coder and never needed to think about these things... :)

You can check WB v3's github at https://github.com/stippi/WonderBrush-v3 , here the pertaining paragraph:

16-bit linear and pre-multiplied RGB internal color space. 8 bit sRGB is designed to preserve details in dark and bright colors at the same time so humans can perceive about the same change in brightness going from one value to the next. This is achieved by using a non-uniform distribution of brightness. Non-uniform color values mean you cannot really do any calculations, or if you do, you will get weird artifacts. This is why an RGB color space is used internally, which has a linear change in brightness. To preserve detail, 16 bits per channel are used instead of 8. The colors are also pre-multiplied with the alpha channel. When alpha is zero at a given pixel, the color channels are meaningless when combining with other pixels, and that is what pre-multiplying achieves. This is important in all sorts of calculations, especially filters.

Huh?! :o :)

[dsizzle]

Well, I'm likely describing it incorrectly. But basically ArtPaint uses (1-source alpha) for compositing, which is common. My point is that I think solid color composition might be ok. But the attempts at transparency (like airbrush) are not.

ArtPaint definitely doesn't use premultiplied alpha, but probably should.

[dsizzle]

This is a 50% blue (0, 0, 255, 128) square, on 5 separate layers, offset by 20, 20 each layer: you can see that it does match the expected behavior - only the intersection of all 5 is at or near 100% opacity.

This is the same 50% blue with some brush scribbles, at different "fade" levels, and you can see there's some confusing and bad stuff going on for sure.

[humdingerb]

The one thing with that first example is that you need to use quite lot of layers to paint...

[dsizzle]

ok, I understand now. Yes, it's very broken.

1 and 2 are exactly the same color, but 1 is drawn on a separate layer and 2 is drawn on the same layer as the background. Both should look identical. The alpha for 2 is wrong (and therefore also the color is coming out as a wild shade of pink instead of transparent red).

3 and 4 are drawn on separate layers and they overlay properly. But, the bottom square of 4 should overlay here:

• but since they're on the same layer they cannot overlay. I don't think any other tool works this way.

[dsizzle]

The good news is I think I see where the problem is.

[humdingerb]

Uh uh... that sounds like there is "bad news" as well... :)

[dsizzle]

Oh, this ticket already is the bad news! 😀

[dsizzle]

so - in the process of this fix I have this question: Is there any real point to the freehand tool? it is basically the same as the brush tool, but with a hard-edged brush that can only be round. It seems like just a specific subset of the brush tool.

Change my mind. :D

[humdingerb]

Agreed.

However, one thing that I've been missing with the Brush tool, and that'll get worse when the Freehand tool is removed: To get a round brush, you need to set both Width and Height sliders. A bit annoying IMO... It'd be nice to have a checkbox "Lock" under the Height slider. That would disable the Height slider, set its value to the Width slider and keep it in sync with it.

Shall I open a new enhancement ticket for that?

[dsizzle]

Ok, I feel like I've been working on this forever, and so I just wanted to show progress. It's way more consistent:

Before:

After:

this is the same experiment as before:

1 is on a layer above the bg 2 is on the same layer as the bg 3 is on a layer above the bg 4 are 2 squares on the same layer, drawn over each other

I also had to fix all the other drawing tools - in the case of ellipses I had to reimplement them because the existing one doesn't really work with alpha and overdraws itself. The new ellipse still has some artifacts I need to work out - see the weird lines:

...but the new ellipses are antialiased so that's an improvement!

Overall everything draws mostly as expected.

The brush feels like it still has a bit of a "halo" around it. :disappointed:

I changed Airbrush behavior a bit - so if you use an alpha color, when you spray at 100% it will not go over the alpha. I.e., it's like a transparent ink. If you choose a 50% alpha color, it won't spray more than 50% alpha. You can then go over it repeatedly, but in a given stroke it won't go over the alpha - the top and bottom are a single stroke, the middle is multiple overlaid strokes to build up the color and opacity:

The one thing that is broken is the Transparency tool - it makes an ugly edge around strokes. :disappointed: still working on it, although tbh I am not a huge fan of that tool as it isn't intuitive - it is supposed to make the alpha of whatever you paint equal to the alpha of the current color, and it does so in steps, based on the "Speed" setting. The "Speed" setting is also not a percentage but rather it adds or subtracts that much to the alpha of the color you paint over. Very strange...

Anyway, it's getting there!

[humdingerb]

That is some great progress there! I never really understood the transparency tool. That may have been related to trnsparency in general was a bit screwed in ArtPaint (before your apparently successful work on it)... Maybe it could be combined with the Eraser tool, which pretty much does the same thing when the "Color" (bad label, but I can't think of a better one) is set to "Transparent". It only needs another slider "Pressure" (or something) to set how much of the paint gets removed in one go, i.e. how much it is blended with the current background color or made transparent. Plus a "Hardness" slider to fade the rubber edges.

Aaaaanyway... back to fixing the alpha drawing... :)

[dsizzle]

I liked your idea so I made a ticket for enhancing the eraser and ditching the Transparency tool.

These weird ellipse lines are driving me mad tho... :laughing:

[humdingerb]

I have another idea that I'll open a ticket for momentarily

These weird ellipse lines are driving me mad tho...

I suppose it would be difficult to use BeAPI calls like FillEllipse() etc. instead? Those had many eyes on them over the years to find bugs...

[dsizzle]

There are a lot of areas that i wonder about trying to convert to BeAPI calls to simplify, but I think at the moment the way the drawing into layers works might not be compatible. It's probably worth exploring because there are other primitives that would be nice to have, like rounded rectangle. Also changing the line width for shapes might be hard to do with the current implementation.

But I was hoping this alpha fix would be quicker... 🤣