Clamp negative values in camera rgb

[age1988]

They are just garbage , it's a good idea clamp negative values in camera rgb before converting to xyz and rgb working space

[github-actions[bot]]

This issue did not get any activity in the past 30 days and will be closed in 365 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.

[matt-maguire]

This goes against a basic design in darktable: https://darktable.gitlab.io/doc/en/color_management.html#unbounded_colors

Negative values can arise due to noise, but that doesn't mean that there is no signal there. Clamping is a non-linear operation that destroys data, and can impede modules from further down the pixel pipe such as denoising from doing their job properly. As the section in the user manual states, it is preferable to deal with this further down the pipe when mapping to the output device.

[age1988]

Negative values in camera rgb are created by some demosaicing or resize algorithms, black level substraction and false color suppression. They should be clamped before the color matrix conversion and from my experience i don't see any artifacts or negative impact in denoise or gamut compression but only benefits. I've used the channel mixer to test it because it clamps values but a real clamp module would be better.

[matt-maguire]

Why should negative values be clamped before color matrix multiplication? You understand that a matrix multiplication represents a linear transformation that rotates and scales the vector space to a new coortinate system. Just because a coordinate was negative before the transformation, doesn't mean the corresponding coordinate with a new set of basis vectors will be negative. If you implement a non-linear operation like a multipling by a step function before such a linear transformation, you have destroying data, and the operation is not invertible. Small values from your camera sensor are going to contain a mix of signal and noise -- you may do a black level subtration which sends some of those values negative, but if you are going to apply statistical moethods to recover the signal buried inside that noise, you need those negative values for those statistical methods to work. Once you move into a display-refered colour space that is not designed to handle negative values, then it makes sense to think about these sorts of non-linear mappings, but earlier on in the pipline when you are working with linear colour spaces, this doesn't make sense to me.

[johnny-bit]

Your req:

Clamp negative values in camera rgb

darktable manual and philosophy of manipulating data:

In darktable there is no technical requirement for clamping; due to the fact that we are using floating point arithmetics for all processing steps, unbounded values can be passed along like any other value until the final color conversion module makes sure that they are transferred into the most reasonable color within the selected output color space. Therefore darktable avoids clamping as far as the underlying algorithms allow.

Answer:

Additionally:

I've used the channel mixer to test it because it clamps values but a real clamp module would be better.

And we have like 3 competing PRs to stop it from clamping: #4687, #5106 and #4940 :)

[age1988]

Why should negative values be clamped before color matrix multiplication? You understand that a matrix multiplication represents a linear transformation that rotates and scales the vector space to a new coortinate system. Just because a coordinate was negative before the transformation, doesn't mean the corresponding coordinate with a new set of basis vectors will be negative.

Negative values throw off how color matrix work. When a 3x3 matrix is applied to a negative linear value in is in effect reversing the equation, A filter combination that should be ADDING light to balance is now SUBTRACTING light. Any colors channels that cross that boundary will now have the color differences increased as opposed to being brought closer together. This just means to me wrong colors , negatives values in camera rgb should be clipped for a correct conversion.

[johnny-bit]

Fixed in #4940 (and is available in color calibration module)

[simon-77]

Hello and a big thank you to the whole development team, I really enjoy using darktable for my photos.

As I am currently working scientifically with electron microscopy pictures (STEM) for my bachelor thesis I would like to do the final edit in darktable, but I encountered an issue while using it: My source images are TIFF files (grayscale) which contains a fair amount of image information in negative values, ranging from about -0.008 to 0.005

With darktables "exposure" module I am able to expose nicely the positive part of the image (exposure set to +5 EV), but I could not manage to retrive the image information from negative values. Also the "black level correction" only raises the black background to a homogeneous grey.

Is it somehow possible to avoid the clipping of negative values from the source file? Or at least to add a constant value to the image before clipping?

Regards, Simon

[kmilos]

@simon-77 Not really related to what was discussed here, so please open a separate issue. Although, as this is not photographic data, chances for any extra effort are slim, but let's see (make sure you share a sample, via some cloud service if it's a large file). In the meantime, you can normalize the data in Python using e.g. the tifffile package that excels in microscope TIFFs...

[simon-77]

Thank you very much @kmilos for your fast response.

1) Relation to this discussion From my understanding this discussion was about whether or not to clamp negative values of source files. I have exactly this issue, as negative values from my source file were not retrievable in darktable. (In my case the source file was a TIFF instead of a RAW). Maybe I have interpreted something wrong - I apologize

2) Photographic & Scientific Data At some point in the past darktable was presenting it's capability for processing images for scientific purposes - and it is still mentioned in the "about" page. If processing "scientifically aquired images" (as stated in the "about" page) is still an intended use case, then it justifies for raising a major issue about negative pixel information being thrown away.

If otherwise darktable no longer intends to be used for scientific data - which is totally fine - pleas focus on photographic data. My intention was to point this issue out for darktable to improve its scientific use cases..

3) Workaround I have already created a Matlab script for this purpose, but I would still prefer to not use my script in the future, if darktable would support negative values in source images ;)

---

I am greatful to the whole darktable development team for this great photography editing suite. Depending on the intended use cases, offhandedly clamping negative pixel values might be a very bad thing to do.