Segfault in before/after mode

[heckflosse]

pixls.us user XaVal privately reported a crash which I can reproduce

1) get this raw and this pp3 2) Open the raw, apply the pp3 3) Zoom in to 500% 4) enable before/after mode => segfault

I'll try to find the cuplrit.

[ff2000]

For me it crashes immediately when loading the pp3.

Version: 5.7-1396-g15c225120
Branch: newlocallab
Commit: 15c225120
Commit date: 2019-11-14
Compiler: cc 8.3.0
Processor: Intel(R)\ Core(TM)\ i7-2600K\ CPU\ @\ 3.40GHz
System: Linux
Bit depth: 64 bits
Gtkmm: V3.24.1
Lensfun: V0.3.2.0
Build type: RelWithDebInfo
Build flags:  -std=c++11  -Werror=unused-label -Werror=delete-incomplete -Wall -Wuninitialized -Wcast-qual -Wno-deprecated-declarations -Wno-unused-result -fopenmp -Werror=unknown-pragmas -O2 -g -DNDEBUG -ftree-vectorize
Link flags:   
OpenMP support: ON
MMAP support: ON

bt 
#0  0x0000556eff435730 in rtengine::ImProcFunctions::ShrinkAllL(rtengine::wavelet_decomposition const&, float**, int, int, float*, float*, float*, int)
    (this=<optimized out>, WaveletCoeffs_L=..., buffer=<optimized out>, level=0, dir=1, noisevarlum=0x0, madL=0x7f4005a00830, vari=0x7f4005a00700, edge=1) at /usr/lib/gcc/x86_64-pc-linux-gnu/9.2.0/include/xmmintrin.h:934
#1  0x0000556eff43604d in rtengine::ImProcFunctions::_ZN8rtengine15ImProcFunctions18WaveletDenoiseAllLERKNS_21wavelet_decompositionEPfPA3_fS4_ii._omp_fn.0(void) () at ../rtengine/FTblockDN.cc:2615
#2  0x00007f4034578c72 in GOMP_parallel
    (fn=0x556eff435dd0 <rtengine::ImProcFunctions::_ZN8rtengine15ImProcFunctions18WaveletDenoiseAllLERKNS_21wavelet_decompositionEPfPA3_fS4_ii._omp_fn.0(void)>, data=0x7f4005a00510, num_threads=1, flags=0)
    at /var/tmp/portage/sys-devel/gcc-9.2.0-r2/work/gcc-9.2.0/libgomp/parallel.c:171
#3  0x0000556eff43534d in rtengine::ImProcFunctions::WaveletDenoiseAllL(rtengine::wavelet_decomposition const&, float*, float (*) [3], float*, int, int)
    (this=this@entry=0x556f030c7fe8, WaveletCoeffs_L=..., noisevarlum=noisevarlum@entry=0x0, madL=madL@entry=0x7f4005a00830, vari=vari@entry=0x7f4005a00700, edge=edge@entry=1, denoiseNestedLevels=1) at ../rtengine/FTblockDN.cc:2595
#4  0x0000556eff56dcfe in rtengine::ImProcFunctions::_ZN8rtengine15ImProcFunctions10ip_waveletEPNS_8LabImageES2_iRKNS_10procparams13WaveletParamsERKNS_8WavCurveERKNS_17WavOpacityCurveRGERKNS_17WavOpacityCurveBYERKNS_16WavOpacityCurveWERKNS_17WavOpacityCurveWLER3LUTIfEi._omp_fn.0(void) () at ../rtengine/ipwavelet.cc:919
#5  0x00007f4034578c72 in GOMP_parallel
    (fn=0x556eff56c570 <rtengine::ImProcFunctions::_ZN8rtengine15ImProcFunctions10ip_waveletEPNS_8LabImageES2_iRKNS_10procparams13WaveletParamsERKNS_8WavCurveERKNS_17WavOpacityCurveRGERKNS_17WavOpacityCurveBYERKNS_16WavOpacityCurveWERKNS_17WavOpacityCurveWLER3LUTIfEi._omp_fn.0(void)>, data=0x7f4005a00990, num_threads=1, flags=0) at /var/tmp/portage/sys-devel/gcc-9.2.0-r2/work/gcc-9.2.0/libgomp/parallel.c:171
#6  0x0000556eff562aeb in rtengine::ImProcFunctions::ip_wavelet(rtengine::LabImage*, rtengine::LabImage*, int, rtengine::procparams::WaveletParams const&, rtengine::WavCurve const&, rtengine::WavOpacityCurveRG const&, rtengine::WavOpacityCurveBY const&, rtengine::WavOpacityCurveW const&, rtengine::WavOpacityCurveWL const&, LUT<float>&, int) (this=this@entry=0x556f030c7fe8, lab=0x556f0a145020, dst=0x556f0a145020, kall=<optimized out>, 
    kall@entry=0, waparams=..., wavCLVCcurve=..., waOpacityCurveRG=..., waOpacityCurveBY=..., waOpacityCurveW=..., waOpacityCurveWL=..., wavclCurve=..., skip=7) at ../rtengine/ipwavelet.cc:668
#7  0x0000556eff710206 in rtengine::ImProcCoordinator::updatePreviewImage(int, bool) (this=0x556f030c6000, todo=<optimized out>, panningRelatedChange=true) at ../rtengine/improccoordinator.cc:1280
#8  0x0000556eff7184fe in rtengine::ImProcCoordinator::process() (this=0x556f030c6000) at ../rtengine/improccoordinator.cc:1994
#9  0x00007f4035a4baca in sigc::slot0<void>::operator()() const (this=0x556f0a0c38a0) at /var/tmp/portage/dev-cpp/glibmm-2.58.1/work/glibmm-2.58.1/glib/glibmm/thread.cc:61
#10 0x00007f4035a4baca in (anonymous namespace)::call_thread_entry_slot(void*) (data=0x556f0a0c38a0) at /var/tmp/portage/dev-cpp/glibmm-2.58.1/work/glibmm-2.58.1/glib/glibmm/thread.cc:48
#11 0x00007f40366f04fd in g_thread_proxy (data=0x556f090e4540) at ../glib-2.60.6/glib/gthread.c:805
#12 0x00007f4034532408 in start_thread (arg=<optimized out>) at pthread_create.c:486
#13 0x00007f403446131f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

The default profile works fine (Auto matched curve ISO LOW).

/EDIT/ After doing a clean build (saw that I used gcc-8.3.0 to compile but it picked up gcc-9.2.0 libs) I can load the pp3 and it crashes as described. Updated bt:

Core was generated by `/home/karl/src/RawTherapee/build/RelWithDebInfo/rawtherapee'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x000055d68e99fe93 in rtengine::_ZN8rtengine7boxblurEPPfS1_iiib._omp_fn.1(void) () at ../rtengine/boxblur.cc:130
130                     lenp1v = lenv + onev;
[Current thread is 1 (Thread 0x7f2c50e9d700 (LWP 52804))]
(gdb) bt
#0  0x000055d68e99fe93 in rtengine::_ZN8rtengine7boxblurEPPfS1_iiib._omp_fn.1(void) () at ../rtengine/boxblur.cc:130
#1  0x00007f2c73ed5c72 in GOMP_parallel (fn=0x55d68e99fc60 <rtengine::_ZN8rtengine7boxblurEPPfS1_iiib._omp_fn.1(void)>, data=0x7f2c50e99de0, num_threads=1, flags=0)
    at /var/tmp/portage/sys-devel/gcc-9.2.0-r2/work/gcc-9.2.0/libgomp/parallel.c:171
#2  0x000055d68e9a1547 in rtengine::boxblur(float**, float**, int, int, int, bool) (multiThread=<optimized out>, H=<optimized out>, W=<optimized out>, radius=<optimized out>, dst=0x7f2c50e99e20, src=<optimized out>)
    at ../rtengine/boxblur.cc:53
#3  0x000055d68e9a1547 in rtengine::boxblur(float**, float**, int, int, int, bool) (src=<optimized out>, dst=0x7f2c50e99e20, radius=<optimized out>, W=<optimized out>, H=<optimized out>, multiThread=<optimized out>)
    at ../rtengine/boxblur.cc:32
#4  0x000055d68e9a17b6 in rtengine::boxblur(float*, float*, int, int, int, bool) (src=src@entry=0x55d69f42ca20, dst=dst@entry=0x55d6aaf2bb40, radius=radius@entry=2, W=W@entry=56, H=H@entry=1, multiThread=multiThread@entry=false)
    at ../rtengine/boxblur.cc:406
#5  0x000055d68e762825 in rtengine::ImProcFunctions::ShrinkAllAB(rtengine::wavelet_decomposition const&, rtengine::wavelet_decomposition const&, float**, int, int, float*, float, bool, bool, bool, float*, float*, int, float*, bool)
    (this=<optimized out>, WaveletCoeffs_L=..., WaveletCoeffs_ab=..., buffer=<optimized out>, level=0, dir=1, noisevarchrom=0x55d69e5d4c40, noisevar_ab=<optimized out>, useNoiseCCurve=true, autoch=true, denoiseMethodRgb=false, madL=0x7f2c50e9a3b0, variC=0x0, local=0, madaab=0x0, madCalculated=false) at ../rtengine/FTblockDN.cc:2879
#6  0x000055d68e762ad1 in rtengine::ImProcFunctions::_ZN8rtengine15ImProcFunctions19WaveletDenoiseAllABERKNS_21wavelet_decompositionES3_PfPA3_fS4_ifbbbi._omp_fn.0(void) () at ../rtengine/FTblockDN.cc:2676
#7  0x00007f2c73ed5c72 in GOMP_parallel
    (fn=0x55d68e7628a0 <rtengine::ImProcFunctions::_ZN8rtengine15ImProcFunctions19WaveletDenoiseAllABERKNS_21wavelet_decompositionES3_PfPA3_fS4_ifbbbi._omp_fn.0(void)>, data=0x7f2c50e9a070, num_threads=8, flags=0)
    at /var/tmp/portage/sys-devel/gcc-9.2.0-r2/work/gcc-9.2.0/libgomp/parallel.c:171
#8  0x000055d68e7606df in rtengine::ImProcFunctions::WaveletDenoiseAllAB(rtengine::wavelet_decomposition const&, rtengine::wavelet_decomposition const&, float*, float (*) [3], float*, int, float, bool, bool, bool, int)
    (this=this@entry=0x55d690d39fe8, WaveletCoeffs_L=..., WaveletCoeffs_ab=..., noisevarchrom=noisevarchrom@entry=0x55d69e5d4c40, madL=0x7f2c50e9a3b0, variC=<optimized out>, local=0, noisevar_ab=<optimized out>, useNoiseCCurve=true, autoch=true, denoiseMethodRgb=false, denoiseNestedLevels=8) at ../rtengine/FTblockDN.cc:2657
#9  0x000055d68e76faa6 in rtengine::ImProcFunctions::_ZN8rtengine15ImProcFunctions11RGB_denoiseEiPNS_10ImagefloatES2_S2_PfS3_S3_bRKNS_10procparams19DirPyrDenoiseParamsEdRKNS_10NoiseCurveESA_RfSB_._omp_fn.2(void) ()
    at ../rtengine/FTblockDN.cc:1213
#10 0x00007f2c73ed5c72 in GOMP_parallel
    (fn=0x55d68e76edd0 <rtengine::ImProcFunctions::_ZN8rtengine15ImProcFunctions11RGB_denoiseEiPNS_10ImagefloatES2_S2_PfS3_S3_bRKNS_10procparams19DirPyrDenoiseParamsEdRKNS_10NoiseCurveESA_RfSB_._omp_fn.2(void)>, data=0x7f2c50e9a800, num_threads=1, flags=0) at /var/tmp/portage/sys-devel/gcc-9.2.0-r2/work/gcc-9.2.0/libgomp/parallel.c:171
#11 0x000055d68e768c90 in rtengine::ImProcFunctions::RGB_denoise(int, rtengine::Imagefloat*, rtengine::Imagefloat*, rtengine::Imagefloat*, float*, float*, float*, bool, rtengine::procparams::DirPyrDenoiseParams const&, double, rtengine::NoiseCurve const&, rtengine::NoiseCurve const&, float&, float&) (this=this@entry=0x55d690d39fe8, kall=kall@entry=0, src=0x55d6aff56300, dst=0x55d6aff56300, calclum=<optimized out>, 
    calclum@entry=0x55d6ab016000, ch_M=0x55d690d3b95c, max_r=0x55d690d3b914, max_b=0x55d690d3b938, isRAW=true, dnparams=..., expcomp=expcomp@entry=1.979597806930542, noiseLCurve=..., noiseCCurve=..., nresi=@0x7f2c50e9c330: 0, highresi=@0x7f2c50e9c4b0: 4.17586942e-43) at ../rtengine/FTblockDN.cc:856
#12 0x000055d68e73c7f6 in rtengine::Crop::update(int) (this=0x55d6a70c4180, todo=4095) at ../rtengine/imagesource.h:175
#13 0x000055d68e73e714 in rtengine::Crop::fullUpdate() (this=0x55d6a70c4180) at ../rtengine/dcrop.cc:1699
#14 0x00007f2c753a8aca in sigc::slot0<void>::operator()() const (this=0x55d6aa3b9a10) at /var/tmp/portage/dev-cpp/glibmm-2.58.1/work/glibmm-2.58.1/glib/glibmm/thread.cc:61
#15 0x00007f2c753a8aca in (anonymous namespace)::call_thread_entry_slot(void*) (data=0x55d6aa3b9a10) at /var/tmp/portage/dev-cpp/glibmm-2.58.1/work/glibmm-2.58.1/glib/glibmm/thread.cc:48
#16 0x00007f2c7604d4fd in g_thread_proxy (data=0x55d6acf390f0) at ../glib-2.60.6/glib/gthread.c:805
#17 0x00007f2c73e8f408 in start_thread (arg=<optimized out>) at pthread_create.c:486
#18 0x00007f2c73dbe31f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

[ff2000]

1) It already crashes at 200% -- 100% is fine 2) enable "Auto Fill" in "Lens/Geometry" and the crash doesn't happen anymore. /edit: 3) from the original pp3 (disabled Quto Fill) disable Wavelets AND Noise Reduction also fixes the crash. Disabling only one of Wavelets / Noisereduction doesn't help.

[heckflosse]

I disabled Wavelet and Local contrast. Still crashes, but this patch fixes the crash

diff --git a/rtengine/boxblur.cc b/rtengine/boxblur.cc
index 491ffae14..045c7ac3f 100644
--- a/rtengine/boxblur.cc
+++ b/rtengine/boxblur.cc
@@ -33,6 +33,7 @@ void boxblur(float** src, float** dst, int radius, int W, int H, bool multiThrea
 {
     //box blur using rowbuffers and linebuffers instead of a full size buffer

+    radius = rtengine::min(radius, W - 1, H - 1);
     if (radius == 0) {
         if (src != dst) {
 #ifdef _OPENMP
diff --git a/rtengine/cplx_wavelet_level.h b/rtengine/cplx_wavelet_level.h
index 8664606c6..09b4e4a88 100644
--- a/rtengine/cplx_wavelet_level.h
+++ b/rtengine/cplx_wavelet_level.h
@@ -272,7 +272,7 @@ template<typename T> void wavelet_level<T>::SynthesisFilterHaarVertical (const T
         #pragma omp for nowait
 #endif

-        for(int i = 0; i < skip; i++)
+        for(int i = 0; i < std::min(skip, height); i++)
         {
             for(int j = 0; j < width; j++) {
                 dst[width * i + j] = (srcLo[i * width + j] + srcHi[i * width + j]);

Enabled wavelet again. Now it crashes again. Looking

Edit: with wavelet enabled debug build works but release build crashes. Making relwithdebinfo build now...

[ff2000]

I can still make it crash when I pull up NR Luminance slider:

[Switching to Thread 0x7fffc7d13700 (LWP 227226)]
0x0000555555cd558f in LUT<float>::operator[]<float, float, void> (index=<optimized out>, this=<optimized out>) at ../rtengine/LUT.h:434
434         T operator[](V index) const
(gdb) bt
#0  0x0000555555cd558f in LUT<float>::operator[]<float, float, void>(float) const (index=<optimized out>, this=<optimized out>)
    at ../rtengine/LUT.h:434
#1  0x0000555555cd558f in rtengine::ImProcFunctions::_ZN8rtengine15ImProcFunctions11RGB_denoiseEiPNS_10ImagefloatES2_S2_PfS3_S3_bRKNS_10procparams19DirPyrDenoiseParamsEdRKNS_10NoiseCurveESA_RfSB_._omp_fn.10(void) () at ../rtengine/FTblockDN.cc:1620
#2  0x00007ffff503dce5 in gomp_thread_start (xdata=<optimized out>) at /var/tmp/portage/sys-devel/gcc-9.2.0-r2/work/gcc-9.2.0/libgomp/team.c:108
#3  0x00007ffff4fee408 in start_thread (arg=<optimized out>) at pthread_create.c:486
#4  0x00007ffff4f1d31f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

[heckflosse]

@ff2000 Thanks for reporting. Will have a look...

[heckflosse]

@ff2000 Can you please provide the pp3 to reproduce?

[ff2000]

Here is the diff to the OPs pp3:

$ diff -u img00030050.png.out.pp3 img00030050.nef.pp3
--- img00030050.png.out.pp3     2019-11-17 12:56:44.640799475 +0200
+++ img00030050.nef.pp3 2019-11-19 22:27:33.929881336 +0200
@@ -1,9 +1,9 @@
 [Version]
-AppVersion=5.7-213-ge5258e6f6
-Version=346
+AppVersion=5.7-1417-g9d750f836
+Version=347

 [General]
-Rank=-1
+Rank=0
 ColorLabel=0
 InTrash=false

@@ -230,7 +230,7 @@
 Enabled=true
 Enhance=false
 Median=false
-Luma=0
+Luma=22.149999999999999
 Ldetail=50
 Chroma=6.5700000000000003
 Method=Lab
@@ -318,6 +318,11 @@
 CenterX=0
 CenterY=0

+[Locallab]
+Enabled=false
+Nbspot=0
+Selspot=0
+
 [PCVignette]
 Enabled=false
 Strength=0.59999999999999998
@@ -350,9 +355,9 @@
 Contrast=18
 AutoContrast=false
 AutoRadius=false
-DeconvGamma=1
 DeconvRadius=0.47999999999999998
 DeconvRadiusOffset=0.14000000000000001
+DeconvIterCheck=true
 DeconvIterations=5

 [PostResizeSharpening]

It's late and I would have to start my slow PC for accessing Dropbox. If that's not enough I will upload the whole pp3 tomorrow.

Version: 5.7-1417-g9d750f836
Branch: newlocallab
Commit: 9d750f836
Commit date: 2019-11-19
Compiler: cc 9.2.0

[heckflosse]

@ff2000

It's late and I would have to start my slow PC for accessing Dropbox.

Why use Dropbox for uploading a whole (small) pp3?

Just add the .txt suffix and attach it here ;-)

[ff2000]

Sorry, brain was already asleep ;)

img00030050.nef.pp3.txt

[ff2000]

I now got the very same backtrace as here when loading the unmodified pp3 from the OP. It seems now it's crashing at 300% view xD (reproducable!)

[heckflosse]

@ff2000

I can still make it crash when I pull up NR Luminance slider:

Should be fixed with this patch:

diff --git a/rtengine/FTblockDN.cc b/rtengine/FTblockDN.cc
index ea0565b21..121c8189d 100644
--- a/rtengine/FTblockDN.cc
+++ b/rtengine/FTblockDN.cc
@@ -1599,9 +1599,9 @@ BENCHFUN
                                         Color::Lab2RGBLimit(labdn->L[i1], labdn->a[i1], labdn->b[i1], labdn->L[i1], labdn->a[i1], labdn->b[i1], wip, 9000000.f, 1.f + qhighFactor * realred, 1.f + qhighFactor * realblue, width);
                                         for (int j = tileleft; j < tileright; ++j) {
                                             int j1 = j - tileleft;
-                                            float r_ = labdn->L[i1][j1];
-                                            float g_ = labdn->a[i1][j1];
-                                            float b_ = labdn->b[i1][j1];
+                                            float r_ = std::max(0.f, labdn->L[i1][j1]);
+                                            float g_ = std::max(0.f, labdn->a[i1][j1]);
+                                            float b_ = std::max(0.f, labdn->b[i1][j1]);
                                             //inverse gamma standard (slider)
                                             r_ = r_ < 32768.f ? igamcurve[r_] : (Color::gammanf(r_ / 32768.f, igam) * 65535.f);
                                             g_ = g_ < 32768.f ? igamcurve[g_] : (Color::gammanf(g_ / 32768.f, igam) * 65535.f);

I now got the very same backtrace as here when loading the unmodified pp3 from the OP. It seems now it's crashing at 300% view xD (reproducable!)

This one I can not reproduce

[ff2000]

Fix confirmed. Also fixes the issue you couldn't reproduce.

Edit: After reverting the fix I now tried again to reproduce the 300% crash, and initially wasn't successful with every try. But I can again make it always crash by grabbing the NR-Luma slider, pulling it to the right and again back to 0.0. After that it crashes always when enabling before/after with 300% view. (Guess: probably an issue with rounding of float? I build as RelWithDebInfo, probably Debug behaves better?)

[heckflosse]

@ff2000 I will make a RelWithDebInfo build now. Will take a while...

[heckflosse]

@ff2000 I can not reproduce using a RelWithDebInfo build :frowning_face:

[ff2000]

@heckflosse no reason to be sad ;) Just to be sure: You tried with HEAD at c21d1420983cf4832dcfc88e7ce13db073419497 (reverted 04443345649fad73f250526c5672dab1208a2b32)? Btw. (I'm sure you know) you can have as many build directories as you want, so no reason to do a REbuild with different build type, just cd to RawTherapee/build_debug - or RawTherapee/build_RelWithDebInfo.

This issue is IMO solved with 0444334 and can be closed.

[heckflosse]

@ff2000 I already had 3 build directorys (for gcc release and debug and for clang release). Now I have 4 ;-)