search

lehins / hip

Haskell Image Processing Library
116 stars 17 forks source link

ColorSpace's don't actually type check colour spaces (and use wrong conversion functions due to this) #55

Open YellowOnion opened 4 months ago

YellowOnion commented 4 months ago

The current API only captures basic pixel format information, RGB and YCbCr are not colorspaces, they're pixel formats.

CIE 1931 XYZ colorspace & pixel format is missing completely.

Most professional software makes explicit case of tagging data streams with the correct metadata, so that conversion can be done correctly, for example you should always do resampling in linear light.

A YCbCr picture from a JPEG, DVD, and BluRay are all different, JPEG's tend to be in full luma scaling (0-255) and using sRGB gamut and transfer functions, while DVD and BluRay follow TV standard which use a limited luma (16-235), coupled with Rec. 601 and Rec. 709 gammut and transfer functions respectively.

Note that when you correct for the luma limits, these three formats look mostly the same, Rec 709 and sRGB are basically identical, but Rec 601 looks a little blown out if not converted correctly.

This only gets worse when we start to introduce Rec. 2020 UHD, and Rec 2100 HDR, and scRGB. a Rec. 2100 image with a peak brightness 100x more than sRGB needs to be handled with care.

The type should capture pixel format, white point, primaries, and transfer functions, this should help with metadata preservation, and correct display of images in software.

lehins commented 4 months ago

Current master has diverged from the released version of hip significantly. It now uses Color that hopefully handles color spaces much more accurately.

Unfortunately I cannot find time now to finish up and make a release right now, but maybe this summer I will have a little bit more time that I could dedicate to the project and finally make a release

YellowOnion commented 4 months ago

@lehins thanks for the reply, nice! I keep wanting to use Haskell, but find holes, I'm glad to see someone filling those holes.

I had a quick skim over the library, and I see no distinction between full and limited swing in pixel formats, ffmpeg makes the distinction (yuv420p and yuvj420p), and personally I think it makes the most sense at that layer, I assume complex pixel layouts are outside the scope of Color (like 4:2:0 etc), unless you plan on a library that can handle full pixel layout, then maybe that could be put there, I assume fusion can optimize out any abstract Y'CbCr to real (i.e. limited-swing NV12) conversion layers.