Green channel clipping with Super-pixel and No White Balance Process options

[pauloup]

Hi, I came across a clipping issue when testing the new 4.2.3 Beta 1 version.

Intro

I'm staking an Andromeda sequence shoot with my ~Canton~ Canon T3, using the Super-pixel mode, so the elongated stars can be found. I'm also using the new No White Balance Process option, which for the first time let me subtract the background color in Gimp correctly with no color issues. But I wasn't able to avoid clipping the Green channel. I'll tell my experience and show the issue.

Experience

My first time trying the Super-pixels and No White Balance Processing options combined led to a super-green picture, due to the non normalized addition of 2 green subpixels (I believe). I also had all "Color Adjustments" values set to 1 in "RAW settings", "Align RGB Channels in final image" disabled, and "No Background Calibration" selected in "Stacking Settings". The resulting image:

In this and next images, Gimp histogram is computed just for the layer with the 32bit unnedited DSS output. It also has a layer with the solid background color in Subtract mode (picked for each image accordantly), and a solid white layer in Dodge mode to brighten the image for preview purposes. These additional layers don't contribute to the histogram results.

As you can see, the background subtraction wasn't enough to correct the green value, as expected. So I used Gimp Levels filter to divide the Green channel by 2:

It fixed the green value, but I also could see some very subtle magenta tint in the blacks. So not a perfect result for the background subtraction in Gimp (maybe due to rounding errors?). The processed image has also a clipped Green channel, as expected.

I understood that I should get the Green channel right prior in DSS, so I played with the RAW settings and found out that setting Brightness to 0.5 and Blue and Red scale values to 2 did the job:

I expected this would avoid clipping the green channel. But despite the great results from the background subtraction in Gimp, for the first time finally delivering a true colourless background without artefacts, the Green channel was still clipped.

It led me to think the actual clipping is happening in the Super-pixel procedure. So to test that, I disabled the No White Balance Processing option, set all the "Color Adjustments" values back to 1, and got this:

As you can see, the Green channel came out with no clipping, but the background subtraction in Gimp wasn't enough to fix the incorrect white balance, as expected. If I set "Use Camera White Balance", I get a better white balance but equally incorrect background removal results in Gimp. So not the best option either.

Conclusion

I believe the issue is that Super-pixel results are being clipped, even when setting Brightness to 0.5, combined with the No White Balance Processing option. In the end, I would still use the clipped output, since it's the only way to get the perfect background removal in Gimp, and the clipping isn't even noticeable. But I hope this issue can be fixed by DSS.

Thank you for your time, and let me know if there are any thing I could do to help.

[perdrix52]

Paulo Thank you for the detailed report.

PS Canton T3 ??? Or Canon EOS Rebel T3 (i.e. EOS 1100D)?

Please could you try the attached build? DSS-Setup64.zip

[pauloup]

Sorry, I meant Canon T3. I'll try this build and come back as soon as possible.

[perdrix52]

Looking at your images I suspect that you did not redo image registration when you turned on Super-Pixel mode. You MUST re-do the registration when when switch either to or from Super-Pixel mode as the pixels are a different size in Super Pixel mode!

[pauloup]

I did, believe me. I'm using Super-Pixels because it was the only way to make the stars registration possible. I used 10s shutter, so the stars got too elongated to default registration.

Even with Super-Pixels, just less than half of my 300 images got computed in the final stacking process, even trying with different reference frames.

Finally, I tried the build you attached, but I'm still getting the same clipped Green channel, using Brightness 0.5, and Red/Blue scale 2. Would not be the case of multiplying the channels by these coefficients before adding them in the super-pixel operation? I suppose it's been done after, that's why it's clipping.

P.S.: I learnt about the need to re-register images after setting Super-Pixels in the hard way, a couple days ago, when the resulting image showed no stars at all, a total averaged mess.

[perdrix52]

I just ran a SuperPixel Stack of one of my image sets with No White Balance selected and No Background Calibration

Histogram on completion of the stack looks like this:

After adjustment using linked settings and increasing saturation shift to 20% I get this:

Finally I adjust the green down slightly and get this:

Which looks pretty much right to me

[perdrix52]

PS I did NOT adjust any of Brightness, Red Scale or Blue Scale from the default values of 1.0

[pauloup]

I like doing the post in Gimp, because it allows me to create a custom layer for a perfect background subtraction, as I showed in the inicial post. I feel like I don't have this much control with the post processing tools in DSS.

Your example seems right, but could you open the Autosave file in Gimp or another tool to inspect the green channel histogram and check if its clipped compared to the other 2 channels? From my tests, I guess it should be.

Please, refer to the very first picture I atached in this page, which had the same settings you used (no Adjustments and no White Balance). If you look closely to the Green Channel in a logarithmic scale, you'll probably see it has double value compared to the other 2 channels. So the upper half is clipped, just we can't see because it's doubled.

Like if a pixel had value 0.6 brightness (in a 0 to 1 scale) in both green subpixels, after registering with no White Balance and using Super-Pixels, it will ended up being equal to 1, because they are going to be summed to 1.2, which is clipped to 1. That information is lost, and it shows in the Gimp histogram.

You probably won't be able to see it in your example picture because the clipping is occurring in the very highlights, on stars middle, and just for the green channel. So that's not easily perceived in this particular image.

But I can imagine some situations, like very bright nebulas centers and other regions with pixels of >50% brightness in the green channel when this clipping can be a noticeable issue.

Thank you for investing time and effort in this issue. I hope I have been effective in showing the issue and its importance. If not, I may try stacking a more meaningful image to show the clipping impact visually.

[perdrix52]

I opened it in GIMP but I see no histogram displays - how do I get those (you can tell I don't use GIMP much) - these days I use StarTools. Opening the 32-bit image in Photoshop won't show me any histogram data.

[pauloup]

Yeah, Gimp doesn't show it by default. You may do the following:

• Open the image
• Go to Windows menu > Dockable Dialogs > Histogram
• Where it says Value, select Green
• Then select the last icon in the same row for "Logarithmic histogram"

If you stacked without changing the Adjustment values in RAW settings (all equal 1), then it may not be showing any apparent clipping, but the Green channel has actually twice the value it should have, therefore it's clipped for >50% intensities.

To fix the image you need to reduce by half the intensity on the green channel. Or better, you could use Brightness equal 0.5, and Red and Blue scales equal 2 in RAW settings before stacking.

The former is expected to produce clipped results, but adjusting RAW settings would be expected to prevent clipping, but it isn't actually.

[pauloup]

Here's a screencast showing the process https://photos.app.goo.gl/WGWfAxx3cvkGfALs8

[perdrix52]

I just re-read your problem description and realised that you have made a very common and fundamentally flawed assumption: That the sky background is a neutral grey/black.

It just isn't, it's typically a muddy reddish-brown after allowing for atmospheric effects (but could be sodium yellow) so if you white balance your stack for a neutral sky background, then your whites will come out the worng colour (typically green).

See: https://clarkvision.com/articles/color.of.the.night.sky/

As for using No White Balance. Unless using something like StarTools, I don't recommend it. Try turning off NO White Balance and using the default (daylight WB). If you do this with my camera, the Red Channel is multiplied by 2.42 and the Blue channel by 1.35. The reason for this is that the sensors tend to be most sensitive to green light, less sensitive to blue light, and even less for red light. If you select "No White Balance" then the green will dominate (but NOT because there are more green pixels - DSS allows for that).

[IvoJager]

Hi all,

I think I understand what @pauloup is getting at. Assuming a DSLR (or OSC) with a Bayer matrix, Superpixel mode makes one pixel value out of one RGGB patch. Disregarding sub-pixel channel alignment, the final value should be R = R, G = (G1 + G2) / 2, B = B. I think @pauloup is trying to say that DSS is calculating the final value as R = R, G = G1 + G2, B = B ("non normalized addition of 2 green subpixels"). Any subsequent "forced" white balancing will have "accidentally" corrected this error. However, now that WB is turned off, it is no longer corrected for. Depending on when truncating/clipping to unity of a channel takes place, this can result in incorrect/unwanted behavior. Does this make sense and is this a good summary of the problem? (also @perdrix52, please don't link to that website as it is full of nonsense from a mathematical and signal processing point of view!)

[pauloup]

I incorrectly assumed the issue was in the Super-pixel process. After some tests, I see that the green clipping also happens when using No White Balance with Bilinear. So I'm sorry, I got this wrong and I should have tested it more.

But then it lead me to think the issue is actually in the No White Balance option. I don't think the green is clipping because of the white balance coeficients not being applied, otherwise a simple division by 2 in the green channel would not fix the image.

@IvoJager, thank you for taking a look and give your understanding about the issue. Now that I saw the issue also happens without the Super-pixels option, would be reasonable to think that the Bilinear option is also being affected by not accounting for the 2 green pixels for 1 red/1 blue difference, when turning on the No White Balance option?

Anyway, being a white balance coeficients problem or a not normalized green channel, I still do believe that applying the "Color Adjustments" factors before performing a Bilinear debayering or Super-Pixels operation could improve the clipping issue.

[pauloup]

And sorry for deleting my previous comment. It got some things wrong and I thought I would have had still time for deleting it and making a correct one.

[IvoJager]

@pauloup I'm not entirely sure what the problem might be in your instance. I've seen some datasets from DSS that were not white balanced and debayered with bilinear debayering, and AFAIK they behaved as expected. Perhaps you could share the troublesome dataset with us (Google Drive, Dropbox) so we can have a look?

[pauloup]

@perdrix52, I think you are right about the sensibility of sensor for the green channel being higher. Now I remember that I had to mess with highlight restoration options to prevent pink highlights when editing RAW images.

Now I think the green clipping is probably an issue from my camera, and has nothing to do to DSS. I'm really sorry for wasting your time, David. As explained here by the Darktable developers or more summarized here by ItsMeLenny:

Because you have 2 green pixels for every 1 blue and 1 red, the greens are exposed at (roughly) half brightness. Meaning you're left with blue and red in the highlights beyond highlights resulting in the magenta. You either need to clip the blue and the red channels at the highest green point, or reconstruct the green into the blue and red points.

So I think I can close this issue concluding that:

• The green channel has actually twice its correct value and its clipped at 50% the intensity value relatively to the other channels.
• But that's because it is sampled that way by the camera, and not because DSS is handling it incorrectly.
• The best solution would be, I think, to perform highlight reconstruction in DSS, like others RAW editors as Darktable and RawTherapee do. It could restore clipped highlights and increase stars quality. But that's not an issue, just an improvement.

Thank you @perdrix52 and @IvoJager for your time and interest in this issue.

[pauloup]

Here's a better image to showcase the clipping issue, if anyone is interested. The pink star would not be like that if the White Balance was active (to also clip blue and red channels), or if there was some sort of highlight restoration (to recover green channel).

[IvoJager]

If you could share the Autosave TIFF file with us (Google Drive, Dropbox), we would probably be able to give you a definitive answer on whether this is expected behavior or not (I suspect it is and is and may be an artifact of what is being done in The GIMP).

[pauloup]

@IvoJager, sure. Here's a folder with all the files of the test project from my last comment.

The RAW/Light folder has 10 raw light frames I used for stacking. The Edit folder has the filelist TXT saved from DSS, plus the following Autosaves:

• Autosave.tif - Bilinear, No White Balance, Brightness/Red/Blue=1
• Autosave001.tif - Super-Pixels, No White Balance, Brightness/Red/Blue=1
• Autosave002.tif - Bilinear, No White Balance, Brightness=0.5, Red=2, Blue=2
• Autosave003.tif - Super-Pixels, No White Balance, Brightness=0.5, Red=2, Blue=2

I then quickly edited Autosave003 in Edit/Gimp folder to better show the pink highlights/green channel clipping. The full edit:

Thank you for your interest

[IvoJager]

I'm happy to report Autosave.tif behaves as expected in StarTools, with no issues in the highlights. The below image was processed with an auto-white balance and color constancy engaged (e.g. keeps colors stable and intact into the highlights);

[perdrix52]

Regarding clipping of the green channel - because of the scaling that DSS applies and a degree of highlight clipping in the camera it is fairly normal for the green channel to be slightly clipped.

Here's an extract from DeepSkyStacker's trace file:

Using Daylight White Balance. White balance co-efficients being used are 2.280481, 0.939520, 1.269931, 0.000000 Maximum value pixel has value 13257 Saturation level is 10230 Applying linear stretch to raw data. Scale values 15.549558, 6.406158, 8.659080, 6.406158

Notice that the maximum ADU is 13257, but the camera reports saturation as 10230 (about 77% of maximum ADU)

DSS does scaling by multiplying the ADU by 65535/10230 = 6.406158 so any pixels whose ADU lies between 10230 and 13257 will be clipped to 65535. It is my belief that we should be using the camera reported saturation level rather then maximum ADU for determining the scaling, but I am prepared to be persuaded that DCRaw and LibRaw do that incorrectly (the DSS scaling is directly based on that in the LibRaw/DCRaw code). This is the same as the default highlight mode in DCRaw and LibRaw where the scale factor is based of dividing all the WB coefficients by the smallest WB coefficient (green channel) to normalise and then applying the scaling above.

There is an alternate mode in LibRaw/DCRaw (-H 1) which tends to result in pink highlights, where the WB coefficients are divided by the largest WB coefficient (red), and then scaled as above - for the values above this would use scaled WB values of: 1.0, 0.4119, 0.5568 which is then scaled by 65535/10230 to give: 6.406, 3.567, 3.022 (to four significant places).

Obviously if No White Balance Processing is selected then all ADUs are simply scaled by 6.406 (in this case).

[pauloup]

@IvoJager That's impressive! I liked it very much, the highlights preservation, while keeping a nice dynamic range in the middle tones, and pleasing amount of noise in the recovered shadows! I'm going to use it as a reference for my edit process in Gimp, thank you so much for sharing this edit! I wish I could afford StartTools right now, as I'm seeing it's totally worth the investment. The official documentation is also great, I'm sure I can learn a lot from it to improve my editing.

@perdrix52 I'm afraid I miss some technical knowledge to fully understand your last comment. Do you mean that the only way to avoid clipping "pixels whose ADU lies between 10230 and 13257" would it be by using the -H 1+ option from dcraw? I have had mixed results by playing with dcraw +H in the past. Could the Brightness/Red/Blue scale options from RAW settings be of any help? I mean, are these coefficients being applied before the 6.406 scaling clip when using No White Balance Process?

Thank you @perdrix52 and @IvoJager for sharing your time and expertise.

[perdrix52]

Yes, that's indeed what I was saying as those are defined by the camera to be "fully saturated".

The equivalent of DCRaw -H 1 brings all sorts of other issues though :(

[pauloup]

Thank you for this link https://clarkvision.com/articles/color.of.the.night.sky/

It has been a valuable lesson, and corrected many wrong assumptions I have been making, as you pointed out.