Closed Beep6581 closed 9 years ago
Ingo, just tested raw_alloc (without patch_15 and before the commit #101) and nothing
changed OK at 5755, crash at 5756
Reported by iliasgiarimis
on 2015-03-01 23:00:18
To be exact, the size was 9274 X 5755 OK 9274 X 5756 crash.
Reported by iliasgiarimis
on 2015-03-01 23:02:51
Ilias, it's clear that there's a boundary somewhere. My question was if uncropped smaller
vs. cropped bigger works now.
Anyway, maybe there are some allocations in between which don't free the big block.
Can you give this one a try?
Ingo
Reported by heckflosse@i-weyrich.de
on 2015-03-01 23:14:01
Ilias, the latest patches address the 'strange' behaviour, that LMMSE worked when rawimage
was cropped by camconst, but didn't work, when it was not.
Reported by heckflosse@i-weyrich.de
on 2015-03-01 23:17:12
Have I to apply raw_alloc_01 upon revision 12f92... ?
Reported by iliasgiarimis
on 2015-03-01 23:19:59
No conflicts here, just apply. By accident I included a printf for wavelet in the patch,
but shouldn't matter
Ingo
Reported by heckflosse@i-weyrich.de
on 2015-03-01 23:23:51
yes, you have to apply upon ...
Reported by heckflosse@i-weyrich.de
on 2015-03-01 23:24:36
nonsense, I meant you can apply upon, but you don't have to, sorry
Reported by heckflosse@i-weyrich.de
on 2015-03-01 23:29:15
For uncropped smaller vs crop on larger file it's difficult to find any very close ..
I tryed with pentax 645Z which is close enough
Raw size 8384 X 6208 = 52.048 Mp
Raw crop 9268 X 6200 = 51.261 Mp .. no crash
Now compiling with raw_alloc_01 on latest ..
Reported by iliasgiarimis
on 2015-03-01 23:39:36
Yes, that 'should be' close enough...
Reported by heckflosse@i-weyrich.de
on 2015-03-01 23:54:44
And, just to clarify, we don't speak about 'no crash' in the whole toolchain, but about
'no crash' during LMMSE demosaic...
Reported by heckflosse@i-weyrich.de
on 2015-03-02 00:02:00
Yes, no crash during LMMSE in editor mode 100% view (I use fast demosaic for <100% views)
Just finished testing .. Nothing changes with latest+raw_alloc_01 9274 X 5755 is OK,
9274 X 5756 crashes.
Reported by iliasgiarimis
on 2015-03-02 00:15:21
Should I test with the largest available raw i.e. Phase One 80Mp full size, and start
with crops at 53Mp ?
Reported by iliasgiarimis
on 2015-03-02 00:18:34
Ilias, let's continue tomorrow. I've to go to bed now...
Reported by heckflosse@i-weyrich.de
on 2015-03-02 00:21:49
Ilias, I'll try to implement a fall back to an 8-stripes processing. This will reduce
internal memory usage of lmmse to about 1/8. Patch follows probably this evening.
Ingo
Reported by heckflosse@i-weyrich.de
on 2015-03-02 15:01:07
Ilias, with all respect, but this seems so much effort.. Is this effort worth adapting
to 32 bit 4Gb systems in 2015? There are other demosaic methods that can be used on
such systems.
May be Ingo's time is better spent on forefronts of RT? OpenCL, etc.?
Reported by michaelezra000
on 2015-03-02 15:25:38
Ok, I'll wait then :-)
Reported by heckflosse@i-weyrich.de
on 2015-03-02 20:04:53
#117, Michael, because of the strange way the crash occurred (at fixed size, regardless
the free memory), I had the impression that it was just a barrier somewhere in the
code and it would be easy to solve.
If a rewrite of all the algorithm is needed I also vote to stop spending hours on this.
The improvement both in speed and "capacity" is already great.
After all, 32bit systems are less and less as time passes :).
Ingo, No need or desire to continue on this, but just to fill some information gaps
..
- with the 80Mp file I had LMMSE crash at smaller than 9274X5755 raw crops !!.
- the raw alloc 0/01 patched builds crashed even with amaze/IGV on the full raw crop
(80Mp) while the unpatched worked fine (editor mode at 100% view)
P.S. Dcraw 9.23 is waiting :)
Reported by iliasgiarimis
on 2015-03-02 20:49:26
Michael and Ilias, there are two simple approaches to avoid the lmmse out of memory
crashes.
Neither of them costs much coding time:
1.) fall back to igv in out of memory case (that's the simpler of both approaches)
2.) render a black (or white or whatever colour) image in out of memory case
Ingo
Reported by heckflosse@i-weyrich.de
on 2015-03-02 21:03:06
I am fine with IGV but I'd like to also have the opinion of other win32bit users (gaaned
.. ).
BTW I wonder why my vista 32 do not use a disk swap file !!.
Reported by iliasgiarimis
on 2015-03-02 22:30:42
Ok, here's the fall back to igv :-) To be applied on latest committed lmmse patch.
Ingo
Reported by heckflosse@i-weyrich.de
on 2015-03-02 22:50:11
Ingo thanks, works fine in editor .. now remains the crash in queue which occurs just
after demosaic with all demosaicers ..
console capture with editor
RawTherapee, version 4.2.111+LMMSE_fallback
..
Loading Olympus E-M5MarkII image ..
raw crop: 0 0 9274 6932 (provided by camconst.json)
Preprocessing: 1126000 usec
Demosaic Bayer image using method: fast
Demosaicing Bayer data: fast - 1385933 usec
Applying white balance, color correction & sRBG conversion...
..
Preprocessing: 585768 usec
Demosaic Bayer image using method: amaze
Demosaicing Bayer data: amaze - 6266055 usec
..
Demosaic Bayer image using method: igv
Demosaicing Bayer data: igv - 2193767 usec
..
Demosaic Bayer image using method: lmmse
lmmse_interpolate_omp: allocation of big memory block failed, try to get 5 smaller
ones now...
lmmse_interpolate_omp: allocation of 5 small memory blocks failed, falling back to
igv_interpolate...
Demosaicing Bayer data: lmmse - 2043710 usec
..
console capture with queue, (happens with all demosaicers ..)
Loading Olympus E-M5MarkII image from ...
raw crop: 0 0 9274 6932 (provided by camconst.json)
..
Load C:\...OLYMPUS-E-M5II\LMMSE\P2030073.ORF: 1601152 usec
Olympus E-M5MarkII BFKA00114.badpixels not found
Olympus E-M5MarkII.badpixels not found
Preprocessing: 986581 usec
lmmse_interpolate_omp: allocation of big memory block failed, try to get 5 smaller
ones now...
lmmse_interpolate_omp: allocation of 5 small memory blocks failed, falling back to
igv_interpolate...
Demosaicing Bayer data: lmmse - 2332187 usec
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
Reported by iliasgiarimis
on 2015-03-03 14:05:01
Ilias, thanks for testing.
The crash in queue with all demosaicers is a different Issue.
I'll commit soon and then close this Issue if there are no objections.
Ingo
Reported by heckflosse@i-weyrich.de
on 2015-03-03 14:29:40
lmmse_fallback_igv.patch committed to revision 145e275603ea
Reported by heckflosse@i-weyrich.de
on 2015-03-03 23:12:40
FixedPendingConfirmation
Reported by heckflosse@i-weyrich.de
on 2015-03-06 10:54:34
Fixed
re #119, Ilias. I'll look at the dcraw patch at the weekend
Reported by heckflosse@i-weyrich.de
on 2015-03-11 18:04:27
Originally reported on Google Code with ID 2665
Reported by
heckflosse@i-weyrich.de
on 2015-02-11 21:06:05