Nikon Z6 II NEF raw files are processed to dark

[dirkrydvan]

Describe the bug With Darktable 3.8.0 it seems that all Nikon Z6 II raw files are processed to dark. The NEF files in Darktable are very darker than the camera generated JPEG. With Nikon D700 NEF raw files they still are processed well with default settings. For Example Rawtherapee does process Nikon D700 NEF files with same Bright/Dark behavior as Nikon Z6 II NEF files. The preview with default settings are very close to the Bight/Dark balance of the camera processed JPEGS. I tried it with:

• 12bit uncompressed
• 12bit compressed
• 14bit uncompressed
• 14bit compressed

Here are the raw NEF samples with their camera generated #JPEG. https://cloud.augustusburg.org/s/fxgcPqfZj4xrS4b

To Reproduce Download example NEF raw files and JPEG files from https://cloud.augustusburg.org/s/fxgcPqfZj4xrS4b Open the example NEF raw files in Darktable and the JPEG files in a application You want. Compare Bright/Dark balance of NEF in Darktable and JPEG.

Expected behavior With default setting the Bright/Dark balance is nearly to the JPEG and after setting the base-curve to nikon, then should the preview are very close to the camera generated JPEG.

Screenshots

Platform (please complete the following information):

Darktable Version: 3.8.x
OS: Ubuntu Linux LTS 20.4-4

Additional context https://github.com/darktable-org/darktable/issues/6898 related also to this

[PeterWem]

Raw files will be darker than JPEG SOOC. With that said your raw files don't match the older base curves for Nikon. Probably because of Active D-Lighting : Extra High Turn that off.

RawTherapee shows the same as darktable if you don't add auto-matched curve.

[kmilos]

See also https://discuss.pixls.us/t/nef-from-nikon-d810-looks-under-exposed/29739 and https://discuss.pixls.us/t/exposure-compensation-in-filmic-v4/19708/4

[PeterWem]

Also https://discuss.pixls.us/t/creating-3d-cube-luts-for-camera-ooc-styles/30968 may be of interest, if the goal is to copy SOOC JPEG look.

Btw, your camera is missing a noise profile. Take the sample pictures like in this tutorial https://pixls.us/articles/how-to-create-camera-noise-profiles-for-darktable/

[dirkrydvan]

That means the base-courve nikon and nikon-alternative does not fit anymore like it did for D700 before? But why does the RAW Nikon Z6_2 Pictures are all darker than RAW Nikon D700 pictures with the same darktable settings?

[kmilos]

But why does the RAW Nikon Z6_2 Pictures are all darker than RAW Nikon D700 pictures with the same darktable settings?

As explained: Active D-Lighting in the very high setting will significantly under-expose. Get to know your camera.

[dirkrydvan]

Im am suprissed - I use Active D-Lighting also in D700.

[kmilos]

D700 has "low", "normal" and "high", Z 6 II has "low", "normal", "high" and "extra high". What exact amount of exposure compensation is used in each mode for each camera only Nikon knows (or you must do some extensive testing yourself to get to know your camera.)

[PeterWem]

There is no benefit of using Highlight tone priority in Canon when shooting raw. I think the same goes for Nikon. If you want to use base curve with Active D lightning you will need to compensate with exposure module the same amount as Nikon is underexposing. Perhaps as much as 3 stops.

For example with Canon, because I shoot Canon: If I use Highlight tone priority I add +1 in exposure module and activate the base curve. This is because Canon underexposes 1 stop.

You can create your own base curve based on Very high Active D-lightning or with a 3D LUT. The later was quite simple. You just take a bunch of JPEG (AdobeRGB)+raw files, put them in a folder and run the script. Then you have a profile for Very high Active D-lightning you can apply for all your raw files.

Take 20 pictures of different colourfull things, upload them in that thread and I can create the profile for you.

[dirkrydvan]

I check the D-Lighting Option again on Nikon Z6 II and compared with Nikon D700. It is true - with Z6 the camera measures at least one stop different with active D-Lighting "extra high". The result is an underexposed RAW file. This behavior I never seen at D700. Unfortunately I saw no hint in the manufacturer manual.

This article helped me: https://photographylife.com/which-camera-settings-affect-raw-photos#active-d-lighting

7) Active D-Lighting

If you have a Nikon camera, you may have come across the Active D-Lighting setting. At its face, this seems like a JPEG-only setting. And, for the most part, that’s exactly what it is.

Assuming that you don’t change any other settings, your Active D-Lighting has no effect on the RAW data itself. However, it’s a bit more complicated than that.

Depending upon your Active D-Lighting settings, you will change your camera’s metering significantly. I don’t know why Nikon chose to have a JPEG setting affect the metering for RAW photos, but that’s what it does.

At the highest possible Active D-Lighting setting on my camera (Extra High), my metered exposure was a full stop darker than normal! Without Active D-Lighting, I had an exposure of 1/5 second. When I turned it on, the exposure was 1/10 second.

Again, Active D-Lighting does nothing to the RAW data itself. It only affects your camera’s meter. However, the effect is so significant that you need to be very careful to avoid ever setting Active D-Lighting if you shoot in RAW.

[kmilos]

I stopped reading after "Active D-Lighting has no effect on the RAW data itself". 😲

Nonetheless, please consider closing the issue, as there is no bug in dt.

[pybokeh]

Just wanted to report same thing happened for me with Nikon Z5's NEFs (too dark and colors were flat) even though I confirmed both my Z5 (just bought it today) has Active D-Lighting turned off and DarkTable's setting for input color profile was set to "standard colour matrix" for the "input profile" option and "linear Rec2020 RGB" for the "working profile" option. After some trial and error, the following is what I did to closely match the JPEGs:

1. Set the base curve to whatever you like. I chose Fujifilm-like (you can also choose Nikon-like). This significantly improved exposure and similarly matched JPEG exposure.
2. Set velvia color - this brought color back.
3. Set local contrast - this was personal preference but seems to match similar amount of contrast as in the JPEGs.

EDIT: FWIW, forgot to mention, I happen to also have the Fujifilm X-T4 that someone has lent me and its RAW files are rendering fine with DT (looks similar to the JPEGs).

[ptilopteri]

If all you want is for dt to develop the raw's to match the in camera jpg's, why not use those jpg's instead of bothering to develop the raw files. dt will not automagically match the in camera jpgs, period! and is not ment to.

[pybokeh]

@ptilopteri That is understood, crystal clear already. But there is inconsistency when raw files are opened for other camera bodies which dont seem to have this problem. Also, it would be nice to have a good initial starting point. With that said, I just recently discovered DT and it is an awesome piece of software. Being able to manipulate everything and as I already mentioned, with just a few actions, I was able to get the desired results. That is amazing for someone like me that literally bought a camera yesterday and using DT for first time!

[ptilopteri]

the jpg (and the embedded jpg) are made using the manufacturer's proprietary code which is not available to dev's. and every mfgr is different. it is unrealistic to expect matching results in dt.

would you expect the dev's to spend their time trying to match the mfgr's coding rather than developing dt? make requests to the mfgr's to open source their coding to make matching raw to out-of-camera jpg's would be better than what you ask here.

[PeterWem]

https://discuss.pixls.us/t/creating-3d-cube-luts-for-camera-ooc-styles/30968

[rmerren]

I like (much of) what my Nikon Z5 does in-camera when it produces jpegs, but I would prefer to use that as a starting point in darktable rather than have to choose between jpeg and adjusting everything all the time. But since it is proprietary, and since nobody else has done the work already for me to create presets (which is usually the case--thanks to those people!), I have been experimenting to see what gives me a similar look on the stuff I like and skips the stuff I don't like. My aim is to develop presets, but for now I am just trying to document stuff. Here is what I found so far:

• Exposure: +1.8EV on outdoor shots. Less on indoor, but I don't have a standard starting number yet. I would probably have separate indoor and outdoor presets. This will almost always need tweaking, but it is almost always underexposed in raw (which is, technically, a good thing that helps preserve more data).
• Turn on Velvia at the default (25%) setting
• Turn on color contrast and set both sliders to +1.5 (this might be too much saturation for some--if you want it less vivid then use lower numbers here)
• Turn on haze removal to the default (0.2)
• A little local contrast can help make it pop
• For higher ISO shots, raw denoise can help smooth out the graininess

If someone has a favorite preset out there that they use as a starting point, please post it here (or, if this is the wrong place to discuss this stuff, please point out the right place).