Set default black level correction to zero

[jandren]

Is your feature request related to a problem? I recently got myself a A7S iii and its high dynamic range made me more aware about the weird default black level correction in the exposure module. Here is the result when using + 8ev and the default black level correction:

And with black level correction set to zero:

Zero offset is less washed out and the better option here. Both examples has profiled denoise enable with the chroma only preset.

Describe the solution you'd like Do as the tooltip says and do use zero as the default value here. This is at least the right thing to do when using the sigmoid module. filmic should really handle this in its own module if its needed for the log operation and not here.

Additional context Reason for why this tiny offset was added to begin with (log computational stability?): https://github.com/darktable-org/darktable/commit/7ffecde96a2ae164fc5d60a4cd1c07456ad43f1f

Is the stability of the tone equalizer also dependent on this offset? If so, then handle the log(zero) case in that module instead.

[jandren]

Note. I can commit time to make a PR for this if others agree with the proposed change.

[glasl]

Wow, I was noticing this issue about an hour ago. Stumbled upon your report while checking out the code base.

The slider description states: "(...) this should be 0% except if you want a faded look"

I'm using the A7 mark 1.

[TurboGit]

Reason for why this tiny offset was added to begin with (log computational stability?):

This has been done long ago, probably before I came to dt and if I remember correctly it is just to avoid some possible wrong black level and avoiding harsh shadows.

[TurboGit]

The slider description states: "(...) this should be 0% except if you want a faded look"

Well -0.0002 is very close to zero :)

[glasl]

True. The 0.0002 do make a noticeable difference though.

[TurboGit]

The 0.0002 do make a noticeable difference though.

I have tried in many pictures CR3, NEF, DNG and I have yet to see a different between 0 -0.0002, I have even tested with the snapshot and looking with close zoom on darker areas. Maybe this makes a difference on some picture but not something common to my experiences.

[jandren]

Yes the difference is minimal in most cases. But It strikes me as mostly a hack to solve the issue with complaints about crushed shadows in filmic back then.

Noticed that my link to the original commit that introduced the change didn't get attached properly added it to the issue. Its this commit from June 2020 by aurelien https://github.com/darktable-org/darktable/commit/7ffecde96a2ae164fc5d60a4cd1c07456ad43f1f I'm so far unable to find any PR, issue or discussion about the change.

[SoupyGit]

My memory is a bit hazy, but I seem to recall it was done to avoid black clipping before later modules (like input color profile, or perhaps filmic). It shouldnt make a difference on images with low dynamic range, especially ettr, and might not make a difference if you raise exposure a lot, but potentially will make a small difference on high dynamic range. Easiest way to check is with later modules off and clipping indicator on. If possible, perhaps an alternate option would be to set it at 0 by default and have an auto button beside it which automatically sets it to a value that avoids black clipping.

[todd-prior]

Yes the difference is minimal in most cases. But It strikes me as mostly a hack to solve the issue with complaints about crushed shadows in filmic back then.

Noticed that my link to the original commit that introduced the change didn't get attached properly added it to the issue. Its this commit from June 2020 by aurelien 7ffecde I'm so far unable to find any PR, issue or discussion about the change.

I actually have noticed that for me the optimal setting is one to a few scroll wheel clicks to a positive number ie just a very small positive offset applied.. There is some visual threshold evident that you hit and to me the display just becomes markedly sharper and vibrant. It can be as little as one or two or maybe 3 or 4 sometimes. THis can happen without pushing the blacks to a warning level. I dont know if it is just something with my monitor and or my monitor profile but it can be like night and day in terms of a sort of instantaneous shift from a bit washed out to something resembling a really nice dehazing with very little correction ...

[jandren]

I have spent some time on looking closer at this since my original post last week. Trying to answer the following to questions:

1. What is my camera's actual black point?
2. Would this default bias help with avoiding negative RGB values -> out of camera gamut problems?

Here is what I have done to answer that. I first captured dark images (lens cap on and short shutter) on three cameras that I had access to, Canon 5D mark ii, Nikon D750, and Sony A7s iii. I then plotted the histogram per color channel for a set of common iso values, 200, 3200, and 12800. I finally added the black point provided by rawpy* as a grey vertical line and the resulting black level due to the default black level compensation in the exposure module as a purple vertical line.

*libraw python wrapper

Assuming the true black level is the peak of this distribution (this is what for example Siril estimates on a per pixel basis when you included dark frames in your deep sky stacking). Then my answers based on that data for the above questions are:

1. All three cameras are pretty close to the stated black level but fractional adjustments could be made for low iso images and a bit larger for the higher iso values. The default black level compensation will make the used black level estimate worse in all of the studied cases.
2. The applied bias will probably save users from clipping pixels when using low iso values on cameras with modern sensors but any picture taken at a bit higher iso or with a older will still be likely to have dark pixels go negative as the bias is much smaller than the observed black level variation. So this black level bias isn't really much of a general method to safe guard black clipping for later modules in the pipeline as far as I can tell.

@todd-prior would be interesting to analyze one of your raws as well and see how it looks from a signal perspective if you want to send me a black raw file or two.

[jandren]

The slider description states: "(...) this should be 0% except if you want a faded look"

Well -0.0002 is very close to zero :)

Note that it is

-0.0002 * 2^exposure

I.e it's a much more noticable

-0.0512 at +8 ev

[todd-prior]

@todd-prior would be interesting to analyze one of your raws as well and see how it looks from a signal perspective if you want to send me a black raw file or two.

Sadly, I am working from a phone these days. I have been thinking about a new camera for some time. I looked with interest when the Nikon Z6 came out but never pulled the trigger then the Lumix S5II came out then Canon came with R7, now fuji with the X-S20 and Sony the 6700.... I don't really do any planned or targeted shooting so its really things that I see or am doing while out and around or on the bike and or hiking and the phone has been just too easy to carry compared to camera kit... Retirement is coming soonish so I need a hobby.... that will likely make me take the plunge.... THanks for the creative analysis....

[jandren]

@TurboGit and @SoupyGit you might be interested in the extra research I put into the topic.

[HSUfineprint]

I will vote for yes on this. I did find the black is not black sometimes, then it needed additional manual adjustment. It's annoying.

[github-actions[bot]]

This issue has been marked as stale due to inactivity for the last 60 days. It will be automatically closed in 300 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.

[github-actions[bot]]

This issue was closed because it has been inactive for 300 days since being marked as stale. Please check if the newest release or nightly build has it fixed. Please, create a new issue if the issue is not fixed.