search

CarVac / librtprocess

A project to make RawTherapee's processing algorithms more readily available.
Other
51 stars 23 forks source link

Demosaicing crashes #52

Closed Vincent-FA closed 4 years ago

Vincent-FA commented 4 years ago

Hi, I've tried to integrate librtprocess in siril, and I encounter segfaults when trying to debayer. I have probably missed something... Here's the temporary code. You cannot easily test it because the makefile modification is not on the repository, because it's hard to automate, but I did it manually.

I tried with rcd and vng4, it segfaults with both. What am I doing wrong?

0xf5295c50 in rcd_demosaic () at /home/vinvin/siril/deps/librtprocess/src/demosaic/rcd.cc:283
283                             blue[row][col] = CLIP(rgb[2][idx] * 65535.f);
(gdb) p idx
$3 = 15631
(gdb) p row
$4 = 269
(gdb) p col
$5 = 9
Thread 23 "siril" received signal SIGSEGV, Segmentation fault
librtprocess::interpolate_row_redblue (rawData=0x14d7a78, cfarray=0xfffee2b4, ar=0xe18c95e8, ab=0xe6653c88, pg=0xe3f8ac30, cg=0xe3f8e938, ng=0xe3f92640, i=828, width=3906) at /home/vinvin/siril/deps/librtprocess/src/include/bayerhelper.h:76
76                  ab[j] = cg[j] + rb * 0.25f;
(gdb) p cg[j]
$1 = 291.828125
(gdb) p j
$2 = 222
(gdb) p rb
$3 = -34.6629333
(gdb) bt
#0  0xf5284644 in librtprocess::interpolate_row_redblue(float const* const*, unsigned int const (*) [2], float*, float*, float const*, float const*, float const*, int, int) (rawData=0x14d7a78, cfarray=0xfffee2b4, ar=0xe18c95e8, ab=0xe6653c88, pg=0xe3f8ac30, cg=0xe3f8e938, ng=0xe3f92640, i=828, width=3906)
    at /home/vinvin/siril/deps/librtprocess/src/include/bayerhelper.h:76
#1  0xf529b8c8 in _Z13vng4_demosaiciiPKPKfPPfS4_S4_PA2_KjRKSt8functionIFbdEE._omp_fn.1(void) () at /home/vinvin/siril/deps/librtprocess/src/demosaic/vng4.cc:334

Thanks!

CarVac commented 4 years ago

RCD definitely works for me in Filmulator.

It doesn't look like the row or column should be anywhere near out of bounds, and same with the index...

And index doesn't look like it should be out of bounds, either...

Have you tried allocating the arrays differently? I see that you're stacking them one after another by setting their pointers sequentially, and I don't know enough about malloc (read: I don't know anything about malloc) to know if that works.

Vincent-FA commented 4 years ago

No worries, I made a badly allocated buffer, forgotting that it's 3 times the size on output...