search

Beep6581 / RawTherapee

A powerful cross-platform raw photo processing program
https://rawtherapee.com
GNU General Public License v3.0
2.75k stars 313 forks source link

Blend + Flat-Field = purple #2256

Open Beep6581 opened 9 years ago

Beep6581 commented 9 years ago

Originally reported on Google Code with ID 2272

Using "Blend" Highlight Reconstruction + a flat-field image on an overexposed photo
makes whites turn purple in the main preview and in the saved image, but not in the
thumbnail.

http://rawtherapee.com/shared/test_images/pentax_k10d_ff_purple.jpg
http://rawtherapee.com/shared/test_images/pentax_k10d_ff_purple.pef
http://rawtherapee.com/shared/test_images/pentax_k10d_ff_purple.pp3
Use this flat-field: http://rawtherapee.com/shared/test_images/pentax_k10d_flatfield_samyang8mm_f11.pef

Reported by entertheyoni on 2014-02-28 19:35:07

Beep6581 commented 9 years ago 
DrSlony, try to adjust the white level

Reported by michaelezra000 on 2014-02-28 21:58:57

Beep6581 commented 9 years ago 
To work around it, just change Highlight Reconstruction Method, but I didn't open the
issue to work around it. That is not desired behavior, and its especially no good when
you find out about it after saving the image because the thumb looks fine (panorama,
adjust one shot, paste profile to the rest, send to Queue).

Reported by entertheyoni on 2014-02-28 22:25:09

Beep6581 commented 9 years ago 
Flat field is not applied to thumbnails - that is a separate issue:)

Reported by michaelezra000 on 2014-02-28 22:34:33

Beep6581 commented 9 years ago 
There is no need for it to, as long as it doesn't have bugs like this ;]

Reported by entertheyoni on 2014-02-28 22:40:58

Beep6581 commented 9 years ago 
I'll try to figure out, which dcp-profile is applied to the FF-shot. Actually don't
know whether it's the same which is used for the RAW the FF gets applied to or a different
one.

But looking at the code, I detected another problem: Reference gray for FF is picked
by the four center pixels of the FF-shot. Picking from center with high blur radius
(as for vignetting correction) is a good choice, but it can be a problem, when a dust
spot is in center of the image and you want to use the FF for dust removal (small blur
radius)

Will investigate,

Ingo

Reported by heckflosse@i-weyrich.de on 2014-02-28 23:02:04

Beep6581 commented 9 years ago 
But dust normally has no color:)

Reported by michaelezra000 on 2014-02-28 23:18:13

Beep6581 commented 9 years ago 
Hi Michael,

referring to point 1 of #5: If the calculation of the values for each channel of the
FF-File are different from the calculations of the raw file it gets applied to, that
could lead to colour shifts, I guess (only a guess)

referring to point 2 of #5: If there's a dust spot in center of the picture (where
the reference values are picked from) that could lead to correction in wrong direction,
I guess (also only a guess)

Ingo

Reported by heckflosse@i-weyrich.de on 2014-02-28 23:29:35

Beep6581 commented 9 years ago 
Michael, to be more specific: After reading the code, I think it was made for vignetting
correction with flat field. For this case, it's a good choice to pick in the center
of the FF for reference brightness. But when it's used for dust removal and there's
a dust spot in center of the FF (dust spots can be big with large apertures, though
less pronounced) that might lead to wrong corrections.

Ingo

Reported by heckflosse@i-weyrich.de on 2014-02-28 23:37:13

Beep6581 commented 9 years ago 
Hi Ingo, I suppose that FF performance would be best if FF is applied prior DCP. In
such case DCP needs to be applied once. However, there is a nuance. The DCP is normally
made from DNG file that is NOT FF-corrected. 

For #2, I think dust would make a spot darker, and very unlikely would alter the color.
I recall the sampling from the optical center of the FF was made for vignetting correction.
If that is based on 4 pixels average, then this could be sampled from a larger area
to bypass the dust issue.

Reported by michaelezra000 on 2014-02-28 23:41:04

Beep6581 commented 9 years ago 
I was typing while you posted comment 8:)

Reported by michaelezra000 on 2014-02-28 23:42:06

Beep6581 commented 9 years ago 
How about this - if blur radius is <100, use average of 100x100?

Reported by michaelezra000 on 2014-02-28 23:43:27

Beep6581 commented 9 years ago 
Michael:

I don't think that dust in the center of the FF would alter the colour of the picture,
but it would make the picture darker, because it's the reference point for FF (have
a look at line 1278 to 1286 of rawimagesource.cc). And, what's important, I didn't
get this effect right now (only in my thoughts after having a look at the code), but
the effect should be there when there's a dust spot in these four pixels.

Concerning DCP: I think, your suggestion is a good base, but there's some need to discuss
this :-)

Ingo

Reported by heckflosse@i-weyrich.de on 2014-03-01 00:03:21

Beep6581 commented 9 years ago 
See issue 2003

Flatfield requires variable white level to work with highlight reconstruction

Reported by torger@ludd.ltu.se on 2014-03-02 11:52:09

Beep6581 commented 9 years ago 
I have seen this issue forever, but I just used other means for highlight recovery.
 I always use FF by default, so I never found a cause.

D5100

Reported by raceviper13 on 2014-03-03 13:09:27

Beep6581 commented 9 years ago 
how do we get this addressed?  do I send a flatfield, the settings and an image to go
with it?

Reported by raceviper13 on 2014-06-18 12:14:22

Beep6581 commented 9 years ago 
Yeah won't hurt to have more issues to test this on if someone picks it up.
Upload them to filebin.net and link us to them here.

Reported by entertheyoni on 2014-06-24 12:12:51

Beep6581 commented 9 years ago 
I stumbled into this as I made the flat field correction for xtrans yesterday. I'll
have a look at it when I've test files for other sensors too.

Reported by heckflosse@i-weyrich.de on 2014-06-24 13:01:35

Beep6581 commented 9 years ago 
http://filebin.net/ni3spoo14x

sorry for the delay.

Reported by raceviper13 on 2014-06-27 19:37:48

Beep6581 commented 8 years ago

Currently, the clipped image photo's highlights turn purple only when "auto clip control" is enabled.

Thanatomanic commented 5 years ago

With @heckflosse giving an update to the clip control here #4891, it would be great if this issue could be tackled too. Because what's the point of automatic clip control if it gives you purple highlights? :wink:

Edited, here's an image and flat-field file for which you can clearly see the effect: https://filebin.net/q1za6mx6swhygx7v

heckflosse commented 5 years ago

@Thanatomanic As described in the clip control tooltip, the clip control prevents unclipped raw files from getting clipped by applying a flat field. In this case you should also not get purple highlights. If the raw file already has clipped highlights, clip control can't do much about that and can give purple highlights (which is also mentioned in the tooltip). In this case clip control should not be used.

Thanatomanic commented 5 years ago

Ah sorry, didn't read that. Would it be possible that clip control detects if there are highlights and then just automagically sets itself to 0? Or is this wishful thinking?

heckflosse commented 5 years ago

btw: left is without ff, right is with ff and auto clip control: grafik

heckflosse commented 5 years ago

@Thanatomanic

Would it be possible that clip control detects if there are highlights and then just automagically sets itself to 0?

That should be possible.

Thanatomanic commented 5 years ago

Interesting, this is my observation:

image

Are you using the pp3 provided? I see that auto-ca correction is on, which shouldn't be the case. Otherwise, please apply Neutral and then only switch on the flat-field.

heckflosse commented 5 years ago

I confirm the issue when using neutral profile + ff auto clip control

heckflosse commented 5 years ago

I will try to find a solution for this issue when 5.5 is released. Though I can't promise anything...