New layer I + II fast convolution going to be broken.

rageworx commented 9 months ago

New fast layer I+II convolution found memory collapse error. Need to fix it.

rageworx commented 9 months ago

Error caused,

- Processing SRCNN ... ProcessSRCNN( w=760, h=581, d=4, m=3.000000
srcnntest(27473,0x1e46c9ec0) malloc: Incorrect checksum for freed object 0x143029600: probably modified after being freed.
Corrupt value: 0x4fd000004fc
srcnntest(27473,0x1e46c9ec0) malloc: *** set a breakpoint in malloc_error_break to debug
rY:rCb:rCr:rA:[1]    27473 abort

rageworx commented 9 months ago

And ..

srcnntest(27510,0x1e46c9ec0) malloc: *** set a breakpoint in malloc_error_break to debug
rY:rCb:rCr:rA:Process 27510 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
    frame #0: 0x000000018e4e111c libsystem_kernel.dylib`__pthread_kill + 8
libsystem_kernel.dylib`:
->  0x18e4e111c <+8>:  b.lo   0x18e4e113c               ; <+40>
    0x18e4e1120 <+12>: pacibsp
    0x18e4e1124 <+16>: stp    x29, x30, [sp, #-0x10]!
    0x18e4e1128 <+20>: mov    x29, sp
Target 0: (srcnntest) stopped.

rageworx commented 9 months ago

backtrace

* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
  * frame #0: 0x000000018e4e111c libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x000000018e518cc0 libsystem_pthread.dylib`pthread_kill + 288
    frame #2: 0x000000018e428a40 libsystem_c.dylib`abort + 180
    frame #3: 0x000000018e33fb08 libsystem_malloc.dylib`malloc_vreport + 908
    frame #4: 0x000000018e35f24c libsystem_malloc.dylib`malloc_zone_error + 104
    frame #5: 0x000000018e34b238 libsystem_malloc.dylib`free_list_checksum_botch + 40
    frame #6: 0x000000018e33879c libsystem_malloc.dylib`small_free_list_remove_ptr_no_clear + 960
    frame #7: 0x000000018e333d60 libsystem_malloc.dylib`small_malloc_from_free_list + 528
    frame #8: 0x000000018e3334bc libsystem_malloc.dylib`small_malloc_should_clear + 196
    frame #9: 0x000000018e3332ec libsystem_malloc.dylib`szone_malloc_should_clear + 128
    frame #10: 0x000000018e3c5d9c libsystem_c.dylib`__smakebuf + 388
    frame #11: 0x000000018e3d1f5c libsystem_c.dylib`__swsetup + 188
    frame #12: 0x000000018e3b6da0 libsystem_c.dylib`__sfvwrite + 84
    frame #13: 0x000000018e3d1e48 libsystem_c.dylib`fwrite + 152
    frame #14: 0x000000010002d578 srcnntest`fltk_png_default_write_data + 40
    frame #15: 0x000000010002632c srcnntest`fltk_png_write_sig + 88
    frame #16: 0x000000010002d634 srcnntest`fltk_png_write_info_before_PLTE + 44
    frame #17: 0x000000010002d83c srcnntest`fltk_png_write_info + 44
    frame #18: 0x0000000100002a70 srcnntest`savetocolorpng(Fl_RGB_Image*, char const*) + 440
    frame #19: 0x0000000100001c70 srcnntest`saveImgU8(void*, char const*) + 212
    frame #20: 0x0000000100001d68 srcnntest`saveImgF32(void*, char const*) + 140
    frame #21: 0x0000000100009ac0 srcnntest`libsrcnn::doSRCNN(unsigned char const*, unsigned int, unsigned int, unsigned int, float, unsigned char*&, unsigned int&, unsigned char**, unsigned int*) + 640
    frame #22: 0x000000010000b080 srcnntest`ProcessSRCNN(unsigned char const*, unsigned int, unsigned int, unsigned int, float, unsigned char*&, unsigned int&, unsigned char**, unsigned int*) + 892
    frame #23: 0x0000000100003fa8 srcnntest`main + 1360
    frame #24: 0x000000018e19d0e0 dyld`start + 2360

rageworx commented 9 months ago

Suspicion : Convolution99x11() breaks memory layers in random. Convolution99x11 implementation has wrong memory algorithm.

rageworx commented 6 months ago

back trace of macOS clang(llvm-g++)

* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
  * frame #0: 0x00000001892560dc libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x000000018928dcc0 libsystem_pthread.dylib`pthread_kill + 288
    frame #2: 0x0000000189199a40 libsystem_c.dylib`abort + 180
    frame #3: 0x00000001890b0b08 libsystem_malloc.dylib`malloc_vreport + 908
    frame #4: 0x00000001890d024c libsystem_malloc.dylib`malloc_zone_error + 104
    frame #5: 0x00000001890bc238 libsystem_malloc.dylib`free_list_checksum_botch + 40
    frame #6: 0x00000001890a979c libsystem_malloc.dylib`small_free_list_remove_ptr_no_clear + 960
    frame #7: 0x00000001890a6d50 libsystem_malloc.dylib`free_small + 604
    frame #8: 0x00000001005b6548 libsrcnn.dylib`libsrcnn::Convolution99x11(libsrcnn::ImgF32&, libsrcnn::ImgF32*, float const (*) [9][9], float const*, float const (*) [64], float const*) + 1416
    frame #9: 0x00000001005b6b1c libsrcnn.dylib`libsrcnn::doSRCNN(unsigned char const*, unsigned int, unsigned int, unsigned int, float, unsigned char*&, unsigned int&, unsigned char**, unsigned int*) + 1180
    frame #10: 0x00000001005b7094 libsrcnn.dylib`ProcessSRCNN(unsigned char const*, unsigned int, unsigned int, unsigned int, float, unsigned char*&, unsigned int&, unsigned char**, unsigned int*) + 304
    frame #11: 0x0000000100005b48 srcnntest`main + 892
    frame #12: 0x0000000188f0d0e0 dyld`start + 2360

rageworx commented 6 months ago

back trace of Linux ( gcc )

- Loading image : Pictures/castle.jpg
- Image loaded type : JPEG | 960 x 540 x 3 bytes
- Scaling ratio : 2.00
- Filter : Bicubic
- Processing SRCNN ... ProcessSRCNN( w=960, h=540, d=3, m=2.000000
[New Thread 0x7ffff6d16640 (LWP 23924)]
[New Thread 0x7ffff6515640 (LWP 23925)]
[New Thread 0x7ffff5d14640 (LWP 23926)]
malloc(): invalid size (unsorted)

Thread 1 "srcnntest" received signal SIGABRT, Aborted.
__pthread_kill_implementation (no_tid=0, signo=6, threadid=140737350127552) at ./nptl/pthread_kill.c:44
44  ./nptl/pthread_kill.c: No such file or directory.
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737350127552)
    at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140737350127552)
    at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140737350127552, signo=signo@entry=6)
    at ./nptl/pthread_kill.c:89
#3  0x00007ffff7642476 in __GI_raise (sig=sig@entry=6)
    at ../sysdeps/posix/raise.c:26
#4  0x00007ffff76287f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x00007ffff7689676 in __libc_message (action=action@entry=do_abort, 
    fmt=fmt@entry=0x7ffff77dbb77 "%s\n") at ../sysdeps/posix/libc_fatal.c:155
#6  0x00007ffff76a0cfc in malloc_printerr (
    str=str@entry=0x7ffff77debc0 "malloc(): invalid size (unsorted)")
    at ./malloc/malloc.c:5664
#7  0x00007ffff76a40dc in _int_malloc (
    av=av@entry=0x7ffff781ac80 <main_arena>, bytes=bytes@entry=5761)
    at ./malloc/malloc.c:4002
#8  0x00007ffff76a5139 in __GI___libc_malloc (bytes=5761)
    at ./malloc/malloc.c:3329
#9  0x0000555555599fe9 in fltk_png_malloc ()
#10 0x000055555558e6cb in fltk_png_write_start_row ()
#11 0x0000555555593e0a in fltk_png_write_row ()
#12 0x0000555555573d77 in savetopng (imgcached=0x555555678e30, 
    fpath=0x555555618803 "resized_Y.png") at src/test.cpp:255
#13 0x0000555555573282 in saveImgU8 (img=0x7fffffffd640, 
    fname=0x555555618803 "resized_Y.png") at src/debugtool.cpp:133
#14 0x000055555557336e in saveImgF32 (img=0x7fffffffd7f0, 
    fname=0x555555618803 "resized_Y.png") at src/debugtool.cpp:153
#15 0x0000555555576e03 in libsrcnn::doSRCNN (
    refbuff=0x7ffff7308010 "\362\372\374\362\372\374\361\371\373\360\370\372\355\367\370\354\366\367\353\367\367\352\366\366\347\365\365\346\366\365\346\366\365\344\367\365\344\367\365\343\367\365\342\366\364\342\366\364\352\364\375\352\364\375\353\365\376\353\365\376\353\365\376\352\364\375\351\363\374\350\362\373\353\365\376\353\365\376\352\364\375\352\364\375\351\363\374\350\362\373\347\361\372\347\361\372\347\356\376\347\356\376\346\355\375\344\356\372\343\355\371\340\354\370\340\354\370\336\354\365\336\354\365\335\356\366\336\357\366\336\361\367\335\360\366\334\357\365\332\355\363\330\355\362\331\352\361\331\352\362\327\350\360\325\347\361\322\345\363\316\344\362\314\343\365\311\342\366\306\340\367\305\340\373\303\340\376\303\342\377\301\342\377\300\342\377\301\343\377\301\343\377\274\345\377\274\345\377\274", <incomplete sequence \346>..., w=960, h=540, d=3, 
    muliply=2, outbuff=@0x7fffffffdce8: 0x0, outbuffsz=@0x7fffffffdc9c: 0, 
    convbuff=0x7fffffffdcf0, convbuffsz=0x7fffffffdca0) at src/libsrcnn.cpp:727
#16 0x00005555555774d1 in ProcessSRCNN (
    refbuff=0x7ffff7308010 "\362\372\374\362\372\374\361\371\373\360\370\372\355\367\370\354\366\367\353\367\367\352\366\366\347\365\365\346\366\365\346\366\365\344\367\365\344\367\365\343\367\365\342\366\364\342\366\364\352\364\375\352\364\375\353\365\376\353\365\376\353\365\376\352\364\375\351\363\374\350\362\373\353\365\376\353\365\376\352\364\375\352\364\375\351\363\374\350\362\373\347\361\372\347\361\372\347\356\376\347\356\376\346\355\375\344\356\372\343\355\371\340\354\370\340\354\370\336\354\365\336\354\365\335\356\366\336\357\366\336\361\367\335\360\366\334\357\365\332\355\363\330\355\362\331\352\361\331\352\362\327\350\360\325\347\361\322\345\363\316\344\362\314\343\365\311\342\366\306\340\367\305\340\373\303\340\376\303\342\377\301\342\377\300\342\377\301\343\377\301\343\377\274\345\377\274\345\377\274", <incomplete sequence \346>..., w=960, h=540, d=3,

rageworx commented 6 months ago

Error caused by

                        temp[k] += kernel99[k][i][j] \
                                   * src.buff[ rowf[row + i] * width + colf[col + j] ];

may fixe by refer to https://github.com/shuwang127/SRCNN_Cpp/pull/4/files#diff-37c5198e1471b330c34357189347e7754c1fd981572c251afba9357dc828c240R51.
Deleting rowf[] memory caused different size of deallocation error.

rageworx commented 6 months ago

Error fixed,

It was a my fault,
wrong allocations at rowf[] and colf[] as different sizes by width and height.
Correction will pushed.

rageworx commented 6 months ago

reference

https://github.com/shuwang127/SRCNN_Cpp/pull/4/files

rageworx / libsrcnn