Speedup for LMMSE demosaic

[Beep6581]

Originally reported on Google Code with ID 2665

Opened this Issue to get a new Issue number before I start my work at this Issue.
As usual for demosaic speedups I'll make a series of patches, small steps.

Ingo

Reported by heckflosse@i-weyrich.de on 2015-02-11 21:06:05

[Beep6581]

Ingo, just tested raw_alloc (without patch_15 and before the commit #101) and nothing
changed OK at 5755, crash at 5756

Reported by iliasgiarimis on 2015-03-01 23:00:18

[Beep6581]

To be exact, the size was 9274 X 5755 OK 9274 X 5756 crash.

Reported by iliasgiarimis on 2015-03-01 23:02:51

[Beep6581]

Ilias, it's clear that there's a boundary somewhere. My question was if uncropped smaller
vs. cropped bigger works now.

Anyway, maybe there are some allocations in between which don't free the big block.
Can you give this one a try?

Ingo

Reported by heckflosse@i-weyrich.de on 2015-03-01 23:14:01

---

• _Attachment: raw_alloc_01.patch_

[Beep6581]

Ilias, the latest patches address the 'strange' behaviour, that LMMSE worked when rawimage
was cropped by camconst, but didn't work, when it was not.

Reported by heckflosse@i-weyrich.de on 2015-03-01 23:17:12

[Beep6581]

Have I to apply raw_alloc_01 upon revision 12f92... ?

Reported by iliasgiarimis on 2015-03-01 23:19:59

[Beep6581]

No conflicts here, just apply. By accident I included a printf for wavelet in the patch,
but shouldn't matter

Ingo

Reported by heckflosse@i-weyrich.de on 2015-03-01 23:23:51

[Beep6581]

yes, you have to apply upon ...

Reported by heckflosse@i-weyrich.de on 2015-03-01 23:24:36

[Beep6581]

nonsense, I meant you can apply upon, but you don't have to, sorry

Reported by heckflosse@i-weyrich.de on 2015-03-01 23:29:15

[Beep6581]

For uncropped smaller vs crop on larger file it's difficult to find any very close ..

I tryed with pentax 645Z which is close enough 
Raw size 8384 X 6208 = 52.048 Mp
Raw crop 9268 X 6200 = 51.261 Mp .. no crash

Now compiling with raw_alloc_01 on latest ..

Reported by iliasgiarimis on 2015-03-01 23:39:36

[Beep6581]

Yes, that 'should be' close enough...

Reported by heckflosse@i-weyrich.de on 2015-03-01 23:54:44

[Beep6581]

And, just to clarify, we don't speak about 'no crash' in the whole toolchain, but about
'no crash' during LMMSE demosaic...

Reported by heckflosse@i-weyrich.de on 2015-03-02 00:02:00

[Beep6581]

Yes, no crash during LMMSE in editor mode 100% view (I use fast demosaic for <100% views)

Just finished testing .. Nothing changes with latest+raw_alloc_01 9274 X 5755 is OK,
9274 X 5756 crashes.

Reported by iliasgiarimis on 2015-03-02 00:15:21

[Beep6581]

Should I test with the largest available raw i.e. Phase One 80Mp full size, and start
with crops at 53Mp ?

Reported by iliasgiarimis on 2015-03-02 00:18:34

[Beep6581]

Ilias, let's continue tomorrow. I've to go to bed now...

Reported by heckflosse@i-weyrich.de on 2015-03-02 00:21:49

[Beep6581]

Ilias, I'll try to implement a fall back to an 8-stripes processing. This will reduce
internal memory usage of lmmse to about 1/8. Patch follows probably this evening.

Ingo

Reported by heckflosse@i-weyrich.de on 2015-03-02 15:01:07

[Beep6581]

Ilias, with all respect, but this seems so much effort.. Is this effort worth adapting
to 32 bit 4Gb systems in 2015? There are other demosaic methods that can be used on
such systems.
May be Ingo's time is better spent on forefronts of RT? OpenCL, etc.?

Reported by michaelezra000 on 2015-03-02 15:25:38

[Beep6581]

Ok, I'll wait then :-)

Reported by heckflosse@i-weyrich.de on 2015-03-02 20:04:53

[Beep6581]

#117, Michael, because of the strange way the crash occurred (at fixed size, regardless
the free memory), I had the impression that it was just a barrier somewhere in the
code and it would be easy to solve.

If a rewrite of all the algorithm is needed I also vote to stop spending hours on this.
The improvement both in speed and "capacity" is already great.
After all, 32bit systems are less and less as time passes :).

Ingo, No need or desire to continue on this, but just to fill some information gaps
.. 
- with the 80Mp file I had LMMSE crash at smaller than 9274X5755 raw crops  !!. 
- the raw alloc 0/01 patched builds crashed even with amaze/IGV on the full raw crop
(80Mp) while the unpatched worked fine (editor mode at 100% view)

P.S. Dcraw 9.23 is waiting :)

Reported by iliasgiarimis on 2015-03-02 20:49:26

[Beep6581]

Michael and Ilias, there are two simple approaches to avoid the lmmse out of memory
crashes.

Neither of them costs much coding time:

1.) fall back to igv in out of memory case (that's the simpler of both approaches)
2.) render a black (or white or whatever colour) image in out of memory case

Ingo

Reported by heckflosse@i-weyrich.de on 2015-03-02 21:03:06

[Beep6581]

I am fine with IGV but I'd like to also have the opinion of other win32bit users (gaaned
.. ).

BTW I wonder why my vista 32 do not use a disk swap file !!.

Reported by iliasgiarimis on 2015-03-02 22:30:42

[Beep6581]

Ok, here's the fall back to igv :-) To be applied on latest committed lmmse patch.

Ingo

Reported by heckflosse@i-weyrich.de on 2015-03-02 22:50:11

---

• _Attachment: lmmse_fallback_igv.patch_

[Beep6581]

Ingo thanks, works fine in editor .. now remains the crash in queue which occurs just
after demosaic with all demosaicers ..  

console capture with editor

RawTherapee, version 4.2.111+LMMSE_fallback
..
Loading Olympus E-M5MarkII image ..
raw crop: 0 0 9274 6932 (provided by camconst.json)

Preprocessing: 1126000 usec
Demosaic Bayer image using method: fast
Demosaicing Bayer data: fast - 1385933 usec
Applying white balance, color correction & sRBG conversion...
..
Preprocessing: 585768 usec
Demosaic Bayer image using method: amaze
Demosaicing Bayer data: amaze - 6266055 usec
..
Demosaic Bayer image using method: igv
Demosaicing Bayer data: igv - 2193767 usec
..
Demosaic Bayer image using method: lmmse
lmmse_interpolate_omp: allocation of big memory block failed, try to get 5 smaller
ones now...
lmmse_interpolate_omp: allocation of 5 small memory blocks failed, falling back to
igv_interpolate...
Demosaicing Bayer data: lmmse - 2043710 usec
..

console capture with queue, (happens with all demosaicers ..) 

Loading Olympus E-M5MarkII image from ...
raw crop: 0 0 9274 6932 (provided by camconst.json)
..
Load C:\...OLYMPUS-E-M5II\LMMSE\P2030073.ORF: 1601152 usec
Olympus E-M5MarkII BFKA00114.badpixels not found
Olympus E-M5MarkII.badpixels not found
Preprocessing: 986581 usec
lmmse_interpolate_omp: allocation of big memory block failed, try to get 5 smaller
ones now...
lmmse_interpolate_omp: allocation of 5 small memory blocks failed, falling back to
igv_interpolate...
Demosaicing Bayer data: lmmse - 2332187 usec 
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc

Reported by iliasgiarimis on 2015-03-03 14:05:01

[Beep6581]

Ilias, thanks for testing.
The crash in queue with all demosaicers is a different Issue.
I'll commit soon and then close this Issue if there are no objections.

Ingo

Reported by heckflosse@i-weyrich.de on 2015-03-03 14:29:40

[Beep6581]

lmmse_fallback_igv.patch committed to revision 145e275603ea

Reported by heckflosse@i-weyrich.de on 2015-03-03 23:12:40

• Status changed: FixedPendingConfirmation

[Beep6581]

Reported by heckflosse@i-weyrich.de on 2015-03-06 10:54:34

• Status changed: Fixed

[Beep6581]

re #119, Ilias. I'll look at the dcraw patch at the weekend

Reported by heckflosse@i-weyrich.de on 2015-03-11 18:04:27