search

DeadSix27 / waifu2x-converter-cpp

Improved fork of Waifu2X C++ using OpenCL and OpenCV
MIT License
792 stars 86 forks source link

Unbreak build on FreeBSD #215

Closed jbeich closed 4 years ago

jbeich commented 4 years ago

Tested on FreeBSD 11.3 (Clang/libc++ 8.0), 12.0 (Clang/libc++ 6.0), 13.0 (Clang/libc++ 9.0) and explicitly with GCC 8.1/9.1/7.4/6.5/5.5.

kattjevfel commented 4 years ago

This breaks on linux (gcc 9.2.0)

waifu2x-converter-cpp/out [freebsd] » ./waifu2x-converter-cpp -i 11153579.png              
CUDA: GeForce GTX 1080
Processing file [1/1] "/home/katt/build/waifu2x-converter-cpp/out/11153579.png":
Scaling image from 460x460 to 920x920

Step 01/02: Denoising
Proccessing [1/1] slices
Processing block, column (01/01), row (01/01) ...
[1]    331617 segmentation fault (core dumped)  ./waifu2x-converter-cpp -i 11153579.png

gdb:

Reading symbols from /home/katt/build/waifu2x-converter-cpp/out/waifu2x-converter-cpp...

warning: core file may not match specified executable file.
[New LWP 333262]
[New LWP 333268]
[New LWP 333275]
[New LWP 333266]
[New LWP 333273]
[New LWP 333267]
[New LWP 333269]
[New LWP 333272]
[New LWP 333270]
[New LWP 333277]
[New LWP 333271]
[New LWP 333265]
[New LWP 333276]
[New LWP 333274]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Core was generated by `./waifu2x-converter-cpp --log-level 2 -i 11153579'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007fcaf69c3d96 in Buffer::prealloc (this=0x56473955a8a0, conv=0x5647395671d0, env=0x564739551520) at /home/katt/build/waifu2x-converter-cpp/src/Buffer.cpp:336
336                                     cuCtxPushCurrent(dev->context);
[Current thread is 1 (Thread 0x7fcaf21ee400 (LWP 333262))]
/usr/lib/../share/gcc-9.2.0/python/libstdcxx/v6/xmethods.py:731: SyntaxWarning: list indices must be integers or slices, not str; perhaps you missed a comma?
  refcounts = ['_M_refcount']['_M_pi']

backtrace:

#0  0x00007fcaf69c3d96 in Buffer::prealloc (this=0x56473955a8a0, conv=0x5647395671d0, env=0x564739551520) at /home/katt/build/waifu2x-converter-cpp/src/Buffer.cpp:336
#1  0x00007fcaf69adaa4 in w2xc::convertWithModelsBlockSplit (conv=0x5647395671d0, env=0x564739551520, inputPlane_2=..., outputPlane_2=..., models=std::vector of length 7, capacity 8 = {...}, 
    flops=0x5647395671e8, blockSize=512, fmt=w2xc::IMAGE_RGB_F32, log_level=2) at /home/katt/build/waifu2x-converter-cpp/src/convertRoutine.cpp:381
#2  0x00007fcaf69ac824 in w2xc::convertWithModels (conv=0x5647395671d0, env=0x564739551520, inputPlane=..., outputPlane=..., models=std::vector of length 7, capacity 8 = {...}, 
    flops=0x5647395671e8, blockSize=0, fmt=w2xc::IMAGE_RGB_F32, log_level=2) at /home/katt/build/waifu2x-converter-cpp/src/convertRoutine.cpp:69
#3  0x00007fcaf69b231c in apply_denoise (conv=0x5647395671d0, image=..., denoise_level=1, blockSize=0, fmt=w2xc::IMAGE_RGB_F32) at /home/katt/build/waifu2x-converter-cpp/src/w2xconv.cpp:765
#4  0x00007fcaf69b4d5e in w2xconv_convert_mat (conv=0x5647395671d0, image_dst=0x7ffea4511d20, image_src=0x7ffea4511cc0, denoise_level=1, scale=2, blockSize=0, background=..., has_alpha=true, 
    dst_alpha=true) at /home/katt/build/waifu2x-converter-cpp/src/w2xconv.cpp:1882
#5  0x00007fcaf69b642e in w2xconv_convert_file (conv=0x5647395671d0, dst_path=0x56473955ce60 "/home/katt/build/waifu2x-converter-cpp/out/11153579_[L1][x2.00].png", 
    src_path=0x56473973a3a0 "/home/katt/build/waifu2x-converter-cpp/out/11153579", denoise_level=1, scale=2, blockSize=0, imwrite_params=0x7ffea4512170)
    at /home/katt/build/waifu2x-converter-cpp/src/w2xconv.cpp:2252
#6  0x0000564738eff155 in convert_file (info=..., inputName=filesystem::path "11153579", output=filesystem::path "auto") at /home/katt/build/waifu2x-converter-cpp/src/main.cpp:518
#7  0x0000564738f02abd in main (argc=5, argv=0x7ffea4513a48) at /home/katt/build/waifu2x-converter-cpp/src/main.cpp:1059
jbeich commented 4 years ago

@Kattus, I can't reproduce with GCC 9.1.0 on FreeBSD using CPU or OpenCL. Can you check if CLI and the library are linked against libstdc++? Does the crash affect only CUDA?

$ waifu2x-converter-cpp --list-processor
   0: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz     (FMA       ): num_core=8
   1: Intel(R) HD Graphics Skylake Desktop GT2     (OpenCL    ): num_core=24

$ readelf -d waifu2x-converter-cpp | fgrep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libw2xc.so]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_imgcodecs.so.3.4]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_imgproc.so.3.4]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_core.so.3.4]
 0x0000000000000001 (NEEDED)             Shared library: [libthr.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.5]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.7]

$ readelf -d libw2xc.so | fgrep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_imgcodecs.so.3.4]
 0x0000000000000001 (NEEDED)             Shared library: [libthr.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_imgproc.so.3.4]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_core.so.3.4]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.5]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.7]
kattjevfel commented 4 years ago

Yeah I should've done that in my initial post, turns out it only crashes when using CUDA. Here's the other stuff though..

waifu2x-converter-cpp/out [freebsd] » readelf -d waifu2x-converter-cpp | fgrep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libw2xc.so]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_imgcodecs.so.4.1]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_imgproc.so.4.1]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_core.so.4.1]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
waifu2x-converter-cpp/out [freebsd] » readelf -d libw2xc.so | fgrep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_imgcodecs.so.4.1]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_imgproc.so.4.1]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_core.so.4.1]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
jbeich commented 4 years ago

@Kattus, does the following patch help? CUDA runtime doesn't use libstdc++, and CUDA toolkit is not relevant after build. I'd expect a crash if libstdc++fs symbols satisfied references from old libstdc++, not the other way around.

diff --git a/CMakeLists.txt b/CMakeLists.txt
index bc7cd67..18005a2 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -79,7 +79,7 @@ if(APPLE AND CMAKE_CXX_COMPILER_ID MATCHES "^(Apple)?Clang$")
    endif()
 elseif(UNIX OR MINGW)
 # https://gitlab.kitware.com/cmake/cmake/issues/17834
-   if (NOT FILE_SYSTEM_LIB AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 9)
+   if (NOT FILE_SYSTEM_LIB)
        set(FILE_SYSTEM_LIB "stdc++fs")
    endif()
 elseif(MSVC)
kattjevfel commented 4 years ago

It still crashes but with a different stacktrace and debug output: stacktrace:

Stack trace of thread 344678:
#0  0x00007f05a14a68a8 n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x768a8)
#1  0x00007f05a1493002 n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x63002)
#2  0x00007f05a147108f n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x4108f)
#3  0x00007f05a1471428 n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x41428)
#4  0x00007f05a148fb2e n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x5fb2e)
#5  0x00007f05a1490ee0 n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x60ee0)
#6  0x00007f05a148f824 n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x5f824)
#7  0x00007f05a149531c n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x6531c)
#8  0x00007f05a1497d5e n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x67d5e)
#9  0x00007f05a149942e n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x6942e)
#10 0x0000557dcf9e7155 n/a (/home/katt/build/waifu2x-converter-cpp/out/waifu2x-converter-cpp + 0x13155)
#11 0x0000557dcf9eaabd n/a (/home/katt/build/waifu2x-converter-cpp/out/waifu2x-converter-cpp + 0x16abd)
#12 0x00007f059e15f153 __libc_start_main (libc.so.6 + 0x27153)
#13 0x0000557dcf9e59de n/a (/home/katt/build/waifu2x-converter-cpp/out/waifu2x-converter-cpp + 0x119de)

gdb:

Reading symbols from /home/katt/build/waifu2x-converter-cpp/out/waifu2x-converter-cpp...

warning: core file may not match specified executable file.
[New LWP 344678]
[New LWP 344687]
[New LWP 344691]
[New LWP 344684]
[New LWP 344685]
[New LWP 344683]
[New LWP 344689]
[New LWP 344682]
[New LWP 344688]
[New LWP 344686]
[New LWP 344690]
[New LWP 344681]
[New LWP 344693]
[New LWP 344692]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Core was generated by `./waifu2x-converter-cpp -i 11153579'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f05a14a68a8 in Buffer::get_write_ptr_cuda (this=0x557dd13ed6c0, env=0x557dd13d3520, devid=0) at /home/katt/build/waifu2x-converter-cpp/src/Buffer.cpp:218
218         cuCtxPushCurrent(dev->context);
[Current thread is 1 (Thread 0x7f059ccd1400 (LWP 344678))]

I made sure I had set -DCMAKE_BUILD_TYPE='Debug' (re-compiled 3 times) so idk why it shows n/a

kattjevfel commented 4 years ago

Ignore everything I previously said, it turned I had CUDA uninstalled without me noticing, and even when compiled without CUDA support, it will try to use it. It has nothing to do with your PR. Very sorry for this.