Nikon COOLPIX B700 magenta highlights related to green channel

[rpittaro]

Windows 10 HP Z220 Workstation ,Xeon processor, AMD graphics card, 12G memory. Apparently the green channel saturates earlier than red or blue in RawTherapee causing the white areas to show as pink. I tried messing with the white points adjustment, but raising it causes the blue and red channels to saturate where the green channel does eliminating all detail, and lowering it has little effect. The effect isn't noticeable on other software Nikon Capture NX-D or Adobe Photoshop express. There is no entry in camconst.json for this camera. Could I mitigate this by adding a profile for this camera and making a white level change to green only in the camconst file? Or is this some other mapping problem in dcraw. The problem is "fixed" by applying highlight reconstruction with color propagation. I enclosed a screen shot from Rawtherapee and one from Nikon Capture NX-D showing the difference in green channel histograms (exposure -1). I would be glad to upload an image if helpful.This may not qualify as a bug but any help would be appreciated. Thanks, Rick

[TooWaBoo]

Activate Highlight reconstruction

[rpittaro]

I am using highlight reconstruction but the "color propagation" method works "better" on this photo. I'm still concerned about how green seems to saturate well before red and blue in Rawtherapee, but not other software. Thanks Rick

[TooWaBoo]

Please provide us the RAW and the .pp3 file

[heckflosse]

@rpittaro please test this camconst.json entry:

    { // Quality C
        "make_model": "NIKON COOLPIX B700",
        "ranges": { "white": 4056 }
    },

[heckflosse]

@rpittaro If the highlights are still pinkish with the camconst entry, remove it and follow this instructions:

https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/camconst.json#L245

To use the formula newWL = BL + (oldWL - BL) / F : BL = 200 and oldWL = 4095

[Beep6581]

@rpittaro please upload several raw files which exhibit this problem. Use https://filebin.net/

[rpittaro]

When making this camconst.json file, do I add this to a copy of the existing file? and where do I put it ?

From: Ingo Weyrich Sent: Saturday, September 01, 2018 2:10 AM To: Beep6581/RawTherapee Cc: rpittaro ; Mention Subject: Re: [Beep6581/RawTherapee] Rawtherapee 5.4-755-g3c54d2ef Nikon B700 green saturation of overexposed image (#4764)

@rpittaro please test this camconst.json entry:

{ // Quality C
    "make_model": "NIKON COOLPIX B700",
    "ranges": { "white": 4056 }
},

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[heckflosse]

@rpittaro Open your existing camconst.json file (but not in notepad, use notepad++) and insert it just before

    { // Quality B, no LENR samples
        "make_model": "Nikon D5",

[rpittaro]

The link to the files is https://filebin.net/yil7c2dxl699zkez If you don’t have it Nikon NX-D is free on the Nikon site (for comparison?) Thanks again for your help. I wonder if it’s a camera limitation that Nikon works around in their software? Regards, Rick

From: Beep6581 Sent: Saturday, September 01, 2018 3:56 AM To: Beep6581/RawTherapee Cc: rpittaro ; Mention Subject: Re: [Beep6581/RawTherapee] Nikon COOLPIX B700 magenta highlights related to green channel (#4764)

@rpittaro please upload several raw files which exhibit this problem. Use https://filebin.net/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[rpittaro]

The first fix got rid of the pink but all the colors saturated early (where the green saturated) and all the detail was lost. FYI the highlight reconstruction in color propagation mode does a pretty good job. I’ll try the next fix. Thanks Again Rick

From: Ingo Weyrich Sent: Saturday, September 01, 2018 2:23 AM To: Beep6581/RawTherapee Cc: rpittaro ; Mention Subject: Re: [Beep6581/RawTherapee] Rawtherapee 5.4-755-g3c54d2ef Nikon B700 green saturation of overexposed image (#4764)

@rpittaro If the highlights are still pinkish with the camconst entry, remove it and follow this instructions:

https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/camconst.json#L245

To use the formula newWL = BL + (oldWL - BL) / F : BL = 200 and oldWL = 4095

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[rpittaro]

Ingo, Sorry, I can’t figure out how to set verbose mode per #L245 ? I’m experimenting with another kind of a work around. Using the exposure control in Nikon NX-D to take the image “out of saturation” converting it to a 16 bit tiff, then opening the tiff in RawTherapee. RawTherapee has some processing options that Nikon NX-D doesn’t have. Regards, Rick

From: Ingo Weyrich Sent: Saturday, September 01, 2018 2:23 AM To: Beep6581/RawTherapee Cc: rpittaro ; Mention Subject: Re: [Beep6581/RawTherapee] Rawtherapee 5.4-755-g3c54d2ef Nikon B700 green saturation of overexposed image (#4764)

@rpittaro If the highlights are still pinkish with the camconst entry, remove it and follow this instructions:

https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/camconst.json#L245

To use the formula newWL = BL + (oldWL - BL) / F : BL = 200 and oldWL = 4095

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[gaaned92]

@rpittaro If you want a quick and conclusive identification of the problem, the only way is as @Beep6581 wrote :

@rpittaro please upload several raw files which exhibit this problem. Use https://filebin.net/

[rpittaro]

Uploaded three files earlier today link https://filebin.net/yil7c2dxl699zkez Thanks, Rick

[Thanatomanic]

@rpittaro I have taken a look at your uploaded files. They are all heavily overexposed, so there will be only so much that you can recover from that. I don't see any unusual behavior though. More camera calibration could be done to provide more details for camconst.json, but the fix does the trick your primary issue (the magenta cast).

Because the G channel clips quite a bit earlier than the R and B channels, there is quite some detail left in those channels. You can bring this out by using Highlight reconstruction -> Colour propagation and then set the Highlight compression to about 85. As far as I know, this is the common approach for RT (and maybe in general).

With some additional tweaking, you can get pretty close to your Nikon NX-D output as shown in your first post. DSCN2671.NRW.pp3.txt (remove the .txt extension)

[rpittaro]

Roel, Thanks for the input. That’s what it seems to be, I’m not sure how the Nikon software and Photoshop express get around it, unless they do highlight reconstruction invisible to the user? Thanks for the input. Rick

From: Roel Baars Sent: Sunday, September 02, 2018 3:02 AM To: Beep6581/RawTherapee Cc: rpittaro ; Mention Subject: Re: [Beep6581/RawTherapee] Nikon COOLPIX B700 magenta highlights related to green channel (#4764)

@rpittaro I have taken a look at your uploaded files. They are all heavily overexposed, so there will be only so much that you can recover from that. I don't see any unusual behavior though. More camera calibration could be done to provide more details for camconst.json, but the fix does the trick your primary issue (the magenta cast).

Because the G channel clips quite a bit earlier than the R and B channels, there is quite some detail left in those channels. You can bring this out by using Highlight reconstruction -> Colour propagation and then set the Highlight compression to about 85. As far as I know, this is the common approach for RT (and maybe in general).

With some additional tweaking, you can get pretty close to your Nikon NX-D output as shown in your first post.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[Beep6581]

@rpittaro please shoot and upload photos from set 1: http://rawpedia.rawtherapee.com/Adding_Support_for_New_Raw_Formats

[rpittaro]

Camera doesn’t have LENR. These are from ISO100, ISO200, ISO400, ISO800, ISO1600 and ISO3200. https://filebin.net/v5437ptzzr13lmo0, Thanks, Rick

From: Beep6581 Sent: Sunday, September 02, 2018 5:33 AM To: Beep6581/RawTherapee Cc: rpittaro ; Mention Subject: Re: [Beep6581/RawTherapee] Nikon COOLPIX B700 magenta highlights related to green channel (#4764)

@rpittaro please shoot and upload photos from sets 1 and 2 (if possible): http://rawpedia.rawtherapee.com/Adding_Support_for_New_Raw_Formats

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[Beep6581]

LENR appears to have been enabled in these photos. I base this on the expected fuzziness at the end of the raw histogram when using LENR and on the metadata:

-TIFF-IFD0-ExifIFD-MakerNotes:NoiseReduction=FPNR
-TIFF-IFD0-ExifIFD-MakerNotes:HighISONoiseReduction=Normal

Black: 200

White: Min-above10-Max

• ISO100 4064-4075-4095
• ISO200 4040-4060-4095
• ISO400 4014-4050-4095
• ISO800 3912-4015-4095
• ISO1600 3756-3980-4095
• ISO3200 3489-3840-4095

The white level in the DNGs is always 4000.

@rpittaro could you upload another set with LENR (or apparently FPNR/high ISO noise reduction as Nikon calls it) disabled?

[Beep6581]

diff --git a/rtengine/camconst.json b/rtengine/camconst.json
index b3758a326..5577d4f5b 100644
--- a/rtengine/camconst.json
+++ b/rtengine/camconst.json
@@ -1420,6 +1420,12 @@ Camera constants:
         }
     },

+    { // Quality B
+        "make_model": "NIKON COOLPIX B700",
+        "dcraw_matrix": [ 14387, -6014, -1299, -1357, 9975, 1616, 467, 1047, 4744 ], // Adobe DNG Converter 10.5 ColorMatrix2
+        "ranges": { "white": 4090 }
+    },
+
     { // Quality B, no LENR samples
         "make_model": "Nikon D5",
         "dcraw_matrix": [ 9200,-3522,-992,-5755,13803,2117,-753,1486,6338 ], // adobe dng_v9.5 d65

@rpittaro I'd say this is a pretty good starting-point default profile for your NIKON COOLPIX B700: NIKON COOLPIX B700.pp3.txt Tweak the curve's dark node as needed. In some cases you can switch the highlight reconstruction method to color propagation and crank highlight compression up to 100.

For reference, I attached the preview JPEG images embedded in your raw files, so you can see that Nikon Capture NX-D is not showing the neutral raw image but doing a lot of behind-the-scenes post-processing.

Embedded preview:

Embedded preview:

My NIKON COOLPIX B700.pp3:

A bit of tweaking: DSCN2671 tm.jpg.out.pp3.txt

[rpittaro]

The camera has three noise reduction settings Low, Medium, High,(no off) nothing mentioned in the manual or in the camera settings about Long Exposure NR, LENR. or FPNR. Should I take the images with “LOW” noise reduction. Normal is the default setting. Thanks very much for this effort, I will make a camcast.json file with these constants.

Regards, Rick From: Beep6581 Sent: Monday, September 03, 2018 12:32 AM To: Beep6581/RawTherapee Cc: rpittaro ; Mention Subject: Re: [Beep6581/RawTherapee] Nikon COOLPIX B700 magenta highlights related to green channel (#4764)

LENR appears to have been enabled in these photos. I base this on the expected fuzziness at the end of the raw histogram when using LENR and on the metadata:

-TIFF-IFD0-ExifIFD-MakerNotes:NoiseReduction=FPNR -TIFF-IFD0-ExifIFD-MakerNotes:HighISONoiseReduction=Normal Black: 200

White: Min-above10-Max

a.. ISO100 4064-4075-4095 b.. ISO200 4040-4060-4095 c.. ISO400 4014-4050-4095 d.. ISO800 3912-4015-4095 e.. ISO1600 3756-3980-4095 f.. ISO3200 3489-3840-4095 The while level in the DNGs is always 4000.

@rpittaro could you upload another set with LENR (or apparently FPNR/high ISO noise reduction as Nikon calls it) disabled?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[TooWaBoo]

Camera NR (or other settings like contrast, saturation...) has no effect on RAW images.

[Beep6581]

@rpittaro well the landscape images had FPNR (which appears to be LENR) turned off, while these white-frames have it turned on. You could try taking one shot with NR set to low and sending it to me.

@TooWaBoo that's not true for long-exposure noise reduction.

[rpittaro]

Beep6581, Will do. I wonder if the camera has “automatic “ LENR on long exposures. I took the saturated pics at 1/2 second exposure. The landscape photos where at “normal” exposure times. I’ll upload a saturated pic at a faster shutter speed with “normal “ NR, and one at low NR. Regards, Rick https://filebin.net/wy6wgtnf871e2s22 From: Beep6581 Sent: Monday, September 03, 2018 2:11 PM To: Beep6581/RawTherapee Cc: rpittaro ; Mention Subject: Re: [Beep6581/RawTherapee] Nikon COOLPIX B700 magenta highlights related to green channel (#4764)

@rpittaro well the landscape images had FPNR (which appears to be LENR) turned off, while these white-frames have it turned on. You could try taking one shot with NR set to low and sending it to me.

@TooWaBoo that's not true for long exposure noise reduction.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[Beep6581]

@rpittaro both of these new ISO400 images have LENR disabled. You might be right about FPNR kicking-in automatically, though I wouldn't expect that to happen at 1/2s. Please upload the whole set with LENR turned off, i.e. try keeping the exposure time faster than 1/30s (just a guess).

By the way, 5260 wasn't completely clipped.

[rpittaro]

Will do when I get home from work . Also I have a general question concerning histogram displays in RawTherapee and other photo software (Capture NX-D ie.) Most cameras have 12 or more bits of pixel data (12 bits for the B700). My assumption is that the demosaicing algorithms are some sort of n by n (or n by m) 2d interpolating filters which would output a least a 12 bit value and if they take advantage of the “averaging” effect from such a filter maybe an extra bit or two. It seems that the histogram displays are limited to 8 bits. Is that true? and why? Thanks, Rick

From: Beep6581 Sent: Tuesday, September 4, 2018 3:54 AM To: Beep6581/RawTherapee Cc: rpittaro ; Mention Subject: Re: [Beep6581/RawTherapee] Nikon COOLPIX B700 magenta highlights related to green channel (#4764)

@rpittaro both of these new ISO400 images have LENR disabled. You might be right about FPNR kicking-in automatically, though I wouldn't expect that to happen at 1/2s. Please upload the whole set with LENR turned off, i.e. try keeping the exposure time faster than 1/30s (just a guess).

By the way, 5260 wasn't completely clipped.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[heckflosse]

@rpittaro

It seems that the histogram displays are limited to 8 bits. Is that true? and why?

well, 2^12 = 4096 means to display a linear 12-bit histogram you would need 4096 pixels width. Of course it can be compressed to smaller size, but then it's not worth more than a 8-bit histogram display

[Beep6581]

I will add to that, that you can generate histograms where every pixel represents one sample using G'MIC. As a histogram image 65535px wide is a bit hard to view, here is an example 1800px wide, 600px high, with the y ceiling at 4000.

gmic colorspace_flowers_16int.tif display_histogram 1800,600,3000,0,65535,1,"cut(i\,0\,4000)"

RawTherapee:

For more info, see: https://discuss.pixls.us/t/view-16-bit-histogram/7648

[rpittaro]

So in all cases the full dynamic range is displayed but its just a matter of display resolution? Regards, Rick

From: Beep6581 Sent: Tuesday, September 4, 2018 10:22 AM To: Beep6581/RawTherapee Cc: rpittaro ; Mention Subject: Re: [Beep6581/RawTherapee] Nikon COOLPIX B700 magenta highlights related to green channel (#4764)

I will add to that, that you can generate histograms where every pixel represents one sample using G'MIC. As a histogram image 65535px wide is a bit hard to view, here is an example 1800px wide, 600px high, with the y ceiling at 4000.

gmic colorspace_flowers_16int.tif display_histogram 1800,600,3000,0,65535,1,"cut(i\,0\,4000)"

RawTherapee:

For more info, see: https://discuss.pixls.us/t/view-16-bit-histogram/7648

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[rpittaro]

https://filebin.net/unwzdk3xwjhnrr85 1/20 seemed to turn LENR off according to the EXIF file. Didn’t know that info was in there. Regards, Rick

From: Beep6581 Sent: Tuesday, September 04, 2018 3:54 AM To: Beep6581/RawTherapee Cc: rpittaro ; Mention Subject: Re: [Beep6581/RawTherapee] Nikon COOLPIX B700 magenta highlights related to green channel (#4764)

@rpittaro both of these new ISO400 images have LENR disabled. You might be right about FPNR kicking-in automatically, though I wouldn't expect that to happen at 1/2s. Please upload the whole set with LENR turned off, i.e. try keeping the exposure time faster than 1/30s (just a guess).

By the way, 5260 wasn't completely clipped.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.