White/black/green rectangles/squares/boxes appear in image

[Beep6581]

Originally reported on Google Code with ID 1084

White rectangles appear in the final image file. The problem does not appear if noise
reduction is not enabled. This has happened with multiple image files and with different
RawTherapee versions.

What steps will reproduce the problem?
1. Open the provided raw file
2. Apply the provided .pp3 file
3. Save the final image, and open it using any image viewer

Branch: default
Version: 4.0.3.0
Changeset: 977e9d748557
Compiler: GCC 4.6.1
Processor: undefined
System: Linux
Bit depth: 32 bits
Gtkmm: V2.24.2
Build type: Release
Build flags: -march=i686 -mtune=generic -O2 -pipe -fstack-protector --param=ssp-buffer-size=4
-D_FORTIFY_SOURCE=2
Link flags:  -Wl,-O1,--sort-common,--as-needed,-z,relro,--hash-style=gnu
OpenMP support: ON
MMAP support: ON

Arch Linux-32

Final image file with the problem:
http://i.imgur.com/kO2rs.jpg

Final image file but without noise reduction,
the rectangles are not visible:
http://i.imgur.com/IHFnD.jpg

Raw image file:
http://panuhorsmalahti.fi/IMG_0674.CR2

.pp3 file (save page as to download):
http://panuhorsmalahti.fi/IMG_0674_noise_reduction.jpg.out.pp3

Reported by nawitus on 2011-10-31 17:05:30

[Beep6581]

Hm, this CR2 plus the PP3 (is it the right one?) converts fine here (?)

Reported by oduis@hotmail.com on 2011-10-31 18:50:38

[Beep6581]

Ok. Here too. So maybe something environment or build specific.
1) Muistitko tehdä? make install
2) Run out of memory (32bit build -> 2G limitation)? Probably not in this case because
then code should crash.
3) How about make debug build without any optimization?
4) Compile without OpenMP.
5) Note: you have quite new compiler Compiler: GCC 4.6.1

Reported by GreatBull69 on 2011-11-01 01:36:25

[Beep6581]

I don't think that any of these will be the root cause.

You say it's only related to noise reduction.
Since Emil is currently reworking his noise reduction code (see issue 1039) I think
it's better to wait till that's done instead of fixing the old code that would be replaced
soon.

Reported by oduis@hotmail.com on 2011-11-01 06:10:50

[Beep6581]

I remember (my issue 1069) when i edit raw i try all methods of noise reduction
But issue 1069 has only open jpg and save it to another location

Reported by oddfish49 on 2011-11-01 08:52:10

[Beep6581]

If you dont want wait you can make some test.
1) save tiff. Still white blocks?
2) change resize. Is block size change or same?
3) rotate 10 degree, block rotate or not?
Are block random or always same place?
These test can use to estimate when block appear in pipeline.

Reported by GreatBull69 on 2011-11-01 12:41:42

[Beep6581]

The problem occured on RawTherapee 3.x also, even when editing a .jpeg file to .jpeg,
like here: http://www.flickr.com/photos/nawitus/6224091423/ (there's one small white
rectangle in the upper-right corner).

I didn't compile rawtherapee myself, I'm using the Arch Linux repositories version.

I have 2 GiB of memory. System memory usage stayed below 1500 MiB during processing,
so lack of memory doesn't apper to be the reason.

"save tiff. Still white blocks?"

Yes.

"2) change resize. Is block size change or same?"

If you double the resolution, the rectangle pixel width and height are doubled. In
essence, they size of the rectangles in relation to the image size is constant.

"Are block random or always same place?"

The white rectangles are always exactly the same.

"3) rotate 10 degree, block rotate or not?"

After rotating the image, the rectangles disappear. 

Reported by nawitus on 2011-11-01 13:14:27

[Beep6581]

The issue in 1083 has a PP3 (unfortunately only a PP3, so still cannot reproduce), and
that one also has all noise reductions enabled. So it fit's with the oberservation
here that it's just on NR. So I'd wait till the new NR is there to see if the problem
is already done with that.

Reported by oduis@hotmail.com on 2011-11-01 18:30:56

[Beep6581]

Issue 1069 has been merged into this issue.

Reported by oduis@hotmail.com on 2011-11-01 18:31:31

[Beep6581]

>"3) rotate 10 degree, block rotate or not?" After rotating the image, the rectangles
disappear. 

Very strange. I was expecting only image is rotated and rectangles stay same. Steps
of image process.
1) some raw enchantments
2) demosaicing
3) HL recovery
4) rotate
5) NR + sharpening
6) resize
If problem is before rotating then image and rectangles should rotated together.

How about try compile by your self? Install mercurial and get latest code.

Reported by GreatBull69 on 2011-11-02 01:28:28

[Beep6581]

I have no problem after update RT to 4.0.4.2 (I can't repeat it now)

Reported by oddfish49 on 2011-11-02 12:21:31

[Beep6581]

Great! Nawitus, is it also fixed on your system with 4.0.4? I'd close the issue then.

Reported by oduis@hotmail.com on 2011-11-03 17:04:28

[Beep6581]

If not, try removing the -ffast-math flag from the RTENGINE_CXX_FLAGS parameter in the
root cmakelist (line 31).

Reported by natureh.510 on 2011-11-03 17:23:06

• Labels added: Priority-Critical, Component-Quality
• Labels removed: Priority-Low

[Beep6581]

I don't see white squares but black ones. I guess it's related!? It's there when enabling
the "Edges" sharpening.

Reported by natureh.510 on 2011-11-05 01:43:34

• Status changed: Accepted

[Beep6581]

Some tools to help debug. Install patch. open problem image in editor. save as jpg.
saving take long time and same time you get 5 tiff files box_*.tiff. Looking files
can estimate where boxes "created".  (simpleprocess.cc)

patch is little ugly. It have lua stuff and some other debug writing, but it should
work. Probably icons from issue 965 need copy too.

Reported by GreatBull69 on 2011-11-05 04:57:48

[Beep6581]

Morning Hombre, enabling the edges that in the dog show still no squares here. It's
difficult to fix for me without a repro :-(  Can you send me the PP3?

Reported by oduis@hotmail.com on 2011-11-05 07:43:40

[Beep6581]

RAWFILE was so kind to pm me a sample file I could repro the squares with.
However it looks again like it's the NR in some image situtation, so I'd vote to wait
till the new NR is done.

Reported by oduis@hotmail.com on 2011-11-06 17:42:59

[Beep6581]

I couldn't reproduce this using the supplied raw and pp3 in the latest self-rolled build.
I tried tweaking all noise and detail sliders, no "luck". However, I did experience
problems with shapes in the past, even 1-2 months ago. I'll keep an eye out.

Reported by entertheyoni on 2011-11-06 18:08:55

[Beep6581]

Is this an issue with OpenMP calls?

Reported by ejm.60657 on 2011-11-06 18:27:29

[Beep6581]

I haven't looked in detail when I it seemed to be related to NR again.

It is enough to have NR enabled, but values to zero. The image has pre-edits that make
it very contrasty, so I my guess it's one of the border bugs. Maybe the gamut map because
it seems it's executed always on first view.

rawfile was a bit picky with supplying the image because it showed persons (personal
rights in germany are pretty strict). He is very helpful though, mayby drop him a PM
in the forum to get the sample.

Reported by oduis@hotmail.com on 2011-11-06 18:33:12

[Beep6581]

looks like OpenMP call. size of block can be size of boxes calculated by openMP.

there is 
#pragma omp parallel
and few following
#pragma omp for
#pragma omp for
#pragma omp for

Maybe all "for"s not done by order. 
maybe above can replace something like
#pragma omp parallel for
#pragma omp parallel for
#pragma omp parallel for
so every block is done before next one start.

Reported by GreatBull69 on 2011-11-07 08:28:20

[Beep6581]

No, the actual OpenMP statments are fine, as there is an implied barrier at the end
of each "omp for" sections that you can suppress by specifying "nowait", which is not
the case.

Reported by natureh.510 on 2011-11-07 08:54:29

[Beep6581]

Problem is not in NR function. It is bug in ImProcFunctions::hsv2rgb.
Hint: dont trust comments read code.

Reported by GreatBull69 on 2011-11-20 11:56:10

[Beep6581]

What is the status of this issue?

Reported by entertheyoni on 2011-11-29 20:54:28

[Beep6581]

Please test this patch. Solution found by GreatBull, but optimized a little bit (it
won't change the face of the earth though ;)).

Reported by natureh.510 on 2011-12-03 14:39:47

---

• Attachment: whiteRectangles.patch

[Beep6581]

http://www.rawtherapee.com/shared/test_images/lagow_greensquares_03_48_-2_EV.pef
http://www.rawtherapee.com/shared/test_images/lagow_greensquares_03_48_-2_EV.pef.pp3

I get the same white squares in some photos in the 100% RT preview.
http://i.imgur.com/DCMG1.jpg

The squares are green in the processed photos:
http://i.imgur.com/85asb.jpg

I rendered the same photo a few times, they appeared in the same place each time.
http://i.imgur.com/8fUjK.png

Also note, the Blue Pixels of Doom appear. Turn "Avoid color clipping" off, and both
the blue pixels as well as the large squares disappear. Turn it back on, they reappear.
http://i.imgur.com/6AHRw.jpg

Change working profile from Prophoto to sRGB, both blue pixels and squares disappear.
Change from sRGB back to Prophoto, they stay disappeared, even if you move the preview
around. Reopen the photo and zoom to 100%, they're back.

Turn noise reduction off, the white squares disappear, but the blue pixels remain.
Turn it back on, they reappear.

I noticed there is a red pixel at the heart of two of the three squares in this photo.
You can see it when you turn noise reduction off.
I also noticed that in some photos the white squares wouldn't appear in the RT 100%
preview, but they were there in the processed photo.

Reported by entertheyoni on 2011-12-06 03:41:09

[Beep6581]

Possibly related to issue 1125

Reported by entertheyoni on 2011-12-06 03:42:59

[Beep6581]

They appeared in the same place before as in after the patch in comment 24.

Reported by entertheyoni on 2011-12-06 03:49:54

[Beep6581]

NR on/off my case just change area problem. NR on: rectangle, NR off: one pixel.
#24 is better then my original patch but it do same thing.

New NR (issue 1039) have "same" (=same but different place) bug as #24 patch.

I tried image #25 and it was ok. So #24 patch not installed or not related to #24 patch.

I see blue pixel as described in 1140. So not related to #24 patch.

Maybe RT have many bugs in conversions/fitters. Maybe more stuff like >= is not same
as >

Reported by GreatBull69 on 2011-12-06 06:21:41

[Beep6581]

I updated to latest RT version and Now I can see Green squares too in saved image. Change
Output profile to No ICM so you get black squares (not so bad). Take NR off then rectangles
is gone (maybe one pixel is still broken). There is some random pixels.  All is gone
if take avoid clipping away. So all related to issue 1125.

Looks like 1140 is results of some new inventions.

Reported by GreatBull69 on 2011-12-06 09:50:08

[Beep6581]

#25 Looks like negative RGB values came from somewhere demosaic or HL recovery.

Some interesting console logs can make 978 patch and start rt with (need save image
to see logs):
rawtherapee -D15 2200 1020 10 10
rawtherapee -D15 x y width height

Reported by GreatBull69 on 2011-12-06 14:51:27

[Beep6581]

You might need to use my f22/18mm flatfield image to get the squares to appear:
http://www.rawtherapee.com/shared/test_images/flatfield_F22_18.0_mm.pef

I've been testing, and disabling some of the enabled tools can make the squares disappear,
but its not just one tool, it cam be for example turning off "auto CA", or disabling
the flatfield, or changing the working color space, or turning off "prevent color clipping"...
So it's probably not just any one of them, but the cumulative effect they have on a
pixel in that area where the square appears at.

GreatBull69 were "2200 1020" just example numbers, or do you want me to run RT with
those specific ones?

One of the pixels that the square forms around is at x=1566 y=3271
rawtherapee -D15 1558 3263 16 16
The log: http://min.us/m5HbkMqq1
grep'd from the log:
h:3271 w:1566 r:0.448587 g:-2.424132 b:4.093190
h:3271 w:1566 r:0.448587 g:-2.424132 b:4.093190
h:3271 w:1566 r:0.448587 g:-2.424132 b:4.093190
h:3271 w:1566 L:-14.416652 a:68.538666 b:-88.604736
h:3271 w:1566 L:-11.621348 a:14.734268 b:-20.037079

Reported by entertheyoni on 2011-12-07 18:34:09

[Beep6581]

Can you try building RT with OpenMP turned off to see if that fixes the problem?  Then
we will know for sure whether it is this or something else.

Reported by ejm.60657 on 2011-12-07 18:56:14

[Beep6581]

Emil, this looks more like the effect of negative values for r,g or b. They can cause
some really weird side effects in most filters. Perhaps clipping negative values after
flat field might be sufficient?

Reported by janrinze on 2011-12-07 19:10:52

[Beep6581]

If it's a problem of negative values, I would rather wait until later in the pipeline;
for instance negative R,G,B values can be caused by out of gamut colors in some working
spaces, and it would be nice to allow the user to rescue them rather than clipping
them, eg by using the HSV curves (this works very nicely with the 'bridge' image. 
  I also want negative values to be allowed for NR, which will be moved toward the
end of raw processing from where it is now (but after flat field correction).  I would
suspect the right place is in ImProcFunctions::rgbProc just before Lab conversion,
if this is the correct prescription.

Reported by ejm.60657 on 2011-12-07 19:25:30

[Beep6581]

Reproduced same as in comment 25 but with OMP=OFF:
http://i.imgur.com/NqNq1.jpg
(note: there are Blue Pixels of Doom in that screenshot, just imgur's jpeg chroma subsampling
makes them very hard to see)

Reported by entertheyoni on 2011-12-07 19:32:24

[Beep6581]

#31 Try find center of squares. There is some very "bad" values as it spread to big
square. And long log have also few text which tell "location" where sample is taken.

Try for your example pixel and file:
rawtherapee -Y -D15 1565 3270 2 2 -c flatfield_F22_18.0_mm.pef

new patch with more values coming next.

How about negative L values OK or not? I don't see negative values problem if code
can handle them. Random values are not so cool.

Reported by GreatBull69 on 2011-12-08 01:35:39

[Beep6581]

Some more data experimenting on lagow_greensquares... with RT 4.0.6.3 on Win Vista 32,
4GB ram.

A very strange behaviour of "NoiseReduction" witch gives black boxes when operating
together with "Edges".

Enabling the "NoiseReduction" and even if Luminance and chrominance are set to 0 it
starts processing and if "edges" is also enabled it gives black boxes. If i change
Luminance to 1... the boxes disappear. Boxes disappear also if i disable "edges" or
"NoiseReduction".

Sometimes with the combination of the two gilty operations i take a very dark preview
instead of the boxes.

In every configuration that produces black boxes-dark preview Rt becomes very slow.
Sometimes stops responding (without crash).  

Reported by iliasgiarimis on 2011-12-08 08:44:50

[Beep6581]

FYI: "lua" patch include timer logs to study algorithm performance
rawtherapee -D72
rawtherapee -D71

Reported by GreatBull69 on 2011-12-08 13:07:16

[Beep6581]

I have a similar problem whereby the Edges tool creates black artefacts in highlights.
I happens in preview mode (at 100% only) as well as in the final output. It happens
with sRGB colour space and ProPhoto and using camera standard or icc profiles makes
no difference.

Branch: default
Version: 4.0.6.3
Changeset: 286a06753099
Compiler: GCC 4.6.1
Processor: generic x86
System: Windows
Bit depth: 32 bits
Gtkmm: V2.22.0
Build type: RELEASE
Build flags:  -mtune=generic -O2 -DNDEBUG
Link flags:  -mtune=generic -mwindows -s -Wl,--large-address-aware
OpenMP support: ON
MMAP support: ON

Reported by accountingfgc on 2011-12-08 19:29:44

[Beep6581]

@ comment 37: the boxes disappear if you change completely different settings too, like
prevent color clipping and color profile, so I don't think you should look for blame
in noise reduction itself, it's likely just a cumulative effect.

Reported by entertheyoni on 2011-12-08 22:23:24

[Beep6581]

GreatBull69 why don't you run those tests yourself? Do you not get the black boxes using
my raw+pp3+flatfield?

Reported by entertheyoni on 2011-12-08 22:24:38

[Beep6581]

I not interest to fix these problems only give little more (at least different) help
then "wait for new NR tool". I seen few clever ideas talk about "flight recorder" so
I was just try show: maybe "flight recorder" should be permanent idea, but looks like
it not wanted to use. Back square one: wait for new NR tool.

Reported by GreatBull69 on 2011-12-09 00:59:25

[Beep6581]

I didn't say I don't want to use it, I'm grateful you made it and I hope it helps track
this problem down.

Reported by entertheyoni on 2011-12-09 01:32:41

[Beep6581]

Could you please remind about the "flight recorder" were there previous posts about
it?

Reported by michaelezra000 on 2011-12-09 02:16:27

[Beep6581]

No problem. Your question was good. I hope you get what I think. Critical case and progress
is ... (reread all comments). Similar case to "tomorrow" (e.g. 1140) and progress is
....

"flight recorder" comments are not in this case. I remember see somewhere RT google
code or RT forums. search all cases for "flight" found at least issue 1067. It same
idea even it is little different. Original idea related "blind" windows users who don't
know what is console.

Reported by GreatBull69 on 2011-12-09 02:35:08

[Beep6581]

"flight recorder" forum link http://www.rawtherapee.com/forum/viewtopic.php?f=3&t=3280

Reported by GreatBull69 on 2011-12-09 02:56:11

[Beep6581]

Issue 1189 has been merged into this issue.

Reported by entertheyoni on 2012-01-02 13:52:52

[Beep6581]

Issue 1262 has been merged into this issue.

Reported by entertheyoni on 2012-03-02 02:22:29

[Beep6581]

Issue 955 has been merged into this issue.

Reported by entertheyoni on 2012-03-18 21:38:36

[Beep6581]

Changed the issue name a bit so it's easier to ctrl+f for.

Reported by entertheyoni on 2012-03-18 21:41:51

• Blocked on: #052