viliusle / miniPaint

online image editor
http://viliusle.github.io/miniPaint/
Other
2.59k stars 602 forks source link

Make sure effects and other color management is being calculated in linear RGB space. #353

Open Giwayume opened 1 year ago

Giwayume commented 1 year ago

Here's a fun fact, I found out that most CSS filters are hacks that don't do proper color space conversions that regular image editors do.

For example, css filter: blur(5px). You can see a dark line in the middle where green and red are merged.

image

Here is the same image with a gaussian filter in GIMP.

image

The reason behind this is most images are encoded in the sRGB color space. Working directly on pixels without converting sRGB to linear sRGB first results in mistakes like this. A typical editor workflow is:

  1. Convert your source images from sRGB to linear sRGB
  2. Run your edits/compositing operations using the linear sRGB colors
  3. At the very end, convert linear sRGB back to sRGB for display.

I know the hue-rotate filter also has an issue where it is approximating the rotation in sRGB space instead of properly converting to HSV/HSL first, rotating, then converting back to sRGB. It may be beneficial to implement "slow, but correct" versions of these filters.

Giwayume commented 1 year ago

Note that the alpha blending of brushes/text/etc. also displays a similar issue, though this may be related to pre-multiplied alpha.

GIMP:

image

Minipaint:

image

viliusle commented 1 year ago

I need to do more research on it, and yes, it looks wrong. I just tried to do blur using CSS on chrome - same issue chrome

Giwayume commented 1 year ago

I can't believe there isn't a simple composite operation for "add". I'm having difficulty figuring out how the writers of the spec intended for us to be able to consume the pre-multiplied output of the path drawing operations.

Giwayume commented 1 year ago

Switching to the "display-p3" color space for the canvas in Chrome alleviates the issue somewhat, but it's not completely correct. The effect is just diminished due to the wider range of colors.

image

Giwayume commented 1 year ago

I can't think of a way for the canvas (in realtime) to get the layer alpha blending working in linear space. It can definitely be done with WebGL fairly easily, but that would be quite a significant rewrite.

Specific things like the gaussian blur filter can be fixed, though, since most filters aren't realtime anyways. Maybe provide a toggle in the dialog for gaussian blur for "linear blending", but that no longer applies it as a realtime fx when selected.

Giwayume commented 1 year ago

Here is an example I was testing image compositing in a Canvas renderer vs a WebGL renderer. The WebGL renderer composites images in linear sRGB color space, then gamma corrects at the very end back to non-linear sRGB. This gives the correct result.

Canvas renderer:

image

WebGL renderer:

image

Giwayume commented 1 year ago

https://www.youtube.com/watch?v=LKnqECcg6Gw

viliusle commented 1 year ago

Sadly, as I understand, WebGL use different API than Canvas "2d". So it means rewriting most of miniPaint code.

Giwayume commented 1 year ago

Yes, WebGL is much more complicated to set up. But it is also much more performant and gives the ability to do realtime effects with shaders. May be something to consider.