search

doe300 / VC4C

Compiler for the VC4CL OpenCL implementation
MIT License
117 stars 37 forks source link

Compilation Failure on Arch Linux & distcc #145

Open bronze51 opened 4 years ago

bronze51 commented 4 years ago

I just started using distcc and successfully compiled "hello world", so I thought I move on to bigger things. The build for "vc4c-git" fails and I think it is due to phtread:

Using distcc to compile VC4C. The server is: 

uname -a
Linux heimdallr 5.4.42-1-ARCH #1 SMP PREEMPT Tue May 26 01:48:52 UTC 2020 armv7l GNU/Linux

The host is:

uname -a
Linux walhalla 5.6.15-arch1-1 #1 SMP PREEMPT Wed, 27 May 2020 23:42:26 +0000 x86_64 GNU/Linux

Contents of "~/.cache/yay/vc4c-git/src/VC4C-master/build/CMakeFiles/CMakeError.log"

Performing C SOURCE FILE Test CMAKE_HAVE_LIBC_PTHREAD failed with the following output:
Change Dir: ~/.cache/yay/vc4c-git/src/VC4C-master/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_fcdf3/fast && /usr/bin/make  -f CMakeFiles/cmTC_fcdf3.dir/build.make CMakeFiles/
cmTC_fcdf3.dir/build
make[1]: Entering directory '~/.cache/yay/vc4c-git/src/VC4C-master/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_fcdf3.dir/src.c.o
/usr/lib/distcc/bin/cc   -D_FORTIFY_SOURCE=2 -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -O2 -pipe -fstack-protector
-strong -fno-plt -DCMAKE_HAVE_LIBC_PTHREAD   -o CMakeFiles/cmTC_fcdf3.dir/src.c.o   -c ~/.cache/yay/vc4c-gi
t/src/VC4C-master/build/CMakeFiles/CMakeTmp/src.c
Linking C executable cmTC_fcdf3
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_fcdf3.dir/link.txt --verbose=1
/usr/lib/distcc/bin/cc -D_FORTIFY_SOURCE=2 -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -O2 -pipe -fstack-protector-s
trong -fno-plt -DCMAKE_HAVE_LIBC_PTHREAD  -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now  -rdynamic CMakeFiles/cmTC_f
cdf3.dir/src.c.o  -o cmTC_fcdf3
/usr/bin/ld: CMakeFiles/cmTC_fcdf3.dir/src.c.o: in function `main':
src.c:(.text.startup+0x38): undefined reference to `pthread_create'
/usr/bin/ld: src.c:(.text.startup+0x40): undefined reference to `pthread_detach'
/usr/bin/ld: src.c:(.text.startup+0x48): undefined reference to `pthread_cancel'
/usr/bin/ld: src.c:(.text.startup+0x54): undefined reference to `pthread_join'
collect2: error: ld returned 1 exit status

Complaints of "yay -S vc4c-git"

-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Check for working C compiler: /usr/lib/distcc/bin/cc
-- Check for working C compiler: /usr/lib/distcc/bin/cc - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/lib/distcc/bin/c++
-- Check for working CXX compiler: /usr/lib/distcc/bin/c++ - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- build deb package...
-- Configuring done
-- Generating done
-- Build files have been written to: ~/.cache/yay/vc4c-git/src/VC4CLStdLib-master/build

...

make[1]: *** [CMakeFiles/Makefile2:1957: src/CMakeFiles/VC4CC.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[100%] Linking CXX shared library libcpptest-lite.so
[100%] Built target cpptest-lite
[ 42%] Built target cpptest-lite-project-build
make: *** [Makefile:183: all] Error 2
==> ERROR: A failure occurred in build().
    Aborting...
Error making: vc4c-git
doe300 commented 4 years ago

Does it work if you add -pthread to the list of libraries to link against in https://github.com/doe300/VC4C/blob/master/src/CMakeLists.txt#L74 ?

bronze51 commented 4 years ago

When I download the package form the AUR again to start the build using "yay", that file is overwritten and my addition is lost. I need to research how to tweak the build process or how to start it without overwritting the file ...

doe300 commented 4 years ago 
Performing C SOURCE FILE Test CMAKE_HAVE_LIBC_PTHREAD failed with the following output:

Looking at this error, it looks like it fails to detect pthread in the following lines of the same CMake file.

Do you have the libpthread.so on your system? Is it located in your PATH or LIBRAY_PATH?

k8ieone commented 4 years ago

Hi, I own the Arch package and I pretty much gave up on it. If anyone wants I can transfer the ownership.

bronze51 commented 4 years ago

After unpacking the arch git package with makepkg --nobuild -s. I tried to turn off MULTI_THREADED to bypass the issue with pthread. I then run makepkg --noextract -i but now/still I hit:

distcc[10542] ERROR: compile ~/dev/vc4c-git/src/VC4C-master/src/ProcessUtil.cpp on 192.168.1.190 failed
distcc[10542] (dcc_build_somewhere) Warning: remote compilation of '~/dev/vc4c-git/src/VC4C-master/src/ProcessUtil.cpp' failed, retrying locally
distcc[10542] Warning: failed to distribute ~/dev/vc4c-git/src/VC4C-master/src/ProcessUtil.cpp to 192.168.1.190, running locally instead

I turned off MULTI_THREADED in

./src/VC4C-master/build/src/CMakeFiles/VC4CC.dir/flags.make
./src/VC4C-master/build/src/CMakeFiles/VC4CC.dir/DependInfo.cmake
./src/VC4C-master/build/src/CMakeFiles/VC4C.dir/flags.make
./src/VC4C-master/build/src/CMakeFiles/VC4C.dir/DependInfo.cmake
./src/VC4C-master/CMakeLists.txt
bronze51 commented 4 years ago

libpthread.so is found in \usr\lib but not in PATH.

doe300 commented 4 years ago

I don't really know what is wrong, so all I can do is suggest a few things to test out:

bronze51 commented 4 years ago

I was missing the following dependencies: (Arch package names) "llvm", "opencl-headers" and "ocl-icd"

My next error is:

[  3%] Built target objvc4dis
[  8%] Built target objvc4asm
[ 13%] Built target objvc4base
[ 15%] Built target vc4dis
[ 17%] Built target vc4asm
[ 17%] Built target libvc4asm-static
[ 18%] Built target libvc4asm-shared
[ 18%] Performing update step for 'cpplog-project'
[ 20%] Performing configure step for 'cpplog-project'
-- Configuring done
-- Generating done
-- Build files have been written to: ~/dev/VC4C/build/cpplog/src/cpplog-project-build
[ 20%] Performing build step for 'cpplog-project'
[ 50%] Built target cpplog-static
[100%] Built target cpplog
[ 22%] Built target cpplog-project-build
[ 98%] Built target VC4CC
[ 98%] Linking CXX executable vc4c
~/dev/VC4C/build/src/vc4c: error while loading shared libraries: ~/dev/VC4C/build/src/libVC4CC.so.1.2: unexpected reloc type 0x03
make[2]: *** [src/CMakeFiles/VC4C.dir/build.make:108: src/vc4c] Error 127
make[2]: *** Deleting file 'src/vc4c'
make[1]: *** [CMakeFiles/Makefile2:1963: src/CMakeFiles/VC4C.dir/all] Error 2
make: *** [Makefile:172: all] Error 2

The above error occurs when taking the source from this respository on github. In addition, I can only find get the following more detailed "error" when building the package from the AUR:

~/dev/vc4c-git/src/VC4C-master/src/ProcessUtil.cpp: In function 'int vc4c::runProcess(const string&, std::istream*, std::ostream*, std::ostream*)':
~/dev/vc4c-git/src/VC4C-master/src/ProcessUtil.cpp:250:18: error: ignoring return value of 'ssize_t write(int, const void*, size_t)', declared with attribute warn_unused_result [-Werror=unused-result]
  250 |             write(pipes[STD_IN][WRITE], buffer.data(), static_cast<std::size_t>(in.gcount()));
      |             ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1plus: some warnings being treated as errors
distcc[13536] ERROR: compile (null) on localhost failed
distcc[13500] ERROR: compile ~/dev/vc4c-git/src/VC4C-master/src/ProcessUtil.cpp on localhost failed
make[2]: *** [src/CMakeFiles/VC4CC.dir/build.make:226: src/CMakeFiles/VC4CC.dir/ProcessUtil.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
doe300 commented 4 years ago

So the second part should be an easy fix, I will look at it tomorrow. As a work-around, putting a (void) before the write should silence the error.

The first error seems to be some kind of static/shared linking mismatch, but I don't know any details.

bronze51 commented 4 years ago

No cigar. I thought I put the (void) in the right spot?

~/dev/vc4c-git/src/VC4C-master/src/ProcessUtil.cpp: In function 'int vc4c::runProcess(const string&, std::istream*, std::ostream*, std::ostream*)':
~/dev/vc4c-git/src/VC4C-master/src/ProcessUtil.cpp:250:24: error: ignoring return value of 'ssize_t write(int, const void*, size_t)', declared with attribute warn_unused_result [-Werror=unused-result]
  250 |             (void)write(pipes[STD_IN][WRITE], buffer.data(), static_cast<std::size_t>(in.gcount()));
      |                   ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1plus: some warnings being treated as errors
doe300 commented 4 years ago

Interesting. Yes, the (void) is in the right place, but somehow the compiler does not like it. You could also try with 

auto dummy = write(pipes[STD_IN][WRITE], buffer.data(), static_cast<std::size_t>(in.gcount()));
(void)dummy;
bronze51 commented 4 years ago

That worked! I did get an error but it was ignored:

[E] Sat Jun 20 13:40:00 2020: Errors in precompilation:
[E] Sat Jun 20 13:40:00 2020: distcc[21709] (main) CRITICAL! distcc seems to have invoked itself recursively!
distcc[21708] ERROR: compile (null) on localhost failed with exit code 111
distcc[21707] ERROR: compile (null) on localhost failed with exit code 111

terminate called after throwing an instance of 'vc4c::CompilationError'
  what():  Pre-compilation: Error in precompilation: distcc[21709] (main) CRITICAL! distcc seems to have invoked itself recursively!
distcc[21708] ERROR: compile (null) on localhost failed with exit code 111
distcc[21707] ERROR: compile (null) on localhost failed with exit code 111

[E] Sat Jun 20 13:40:00 2020: Received signal: SIGABRT
k8ieone commented 4 years ago

I separated VC4C, VC4CL and VC4CLStdLib into separate packages but I have bumped into a small problem. While building VC4C it tries to write into the directory containing VC4CLStdLib. Can I somehow tell it not to do that? Or just direct the writing part somewhere else?

[ 80%] Linking CXX executable vc4c
[E] Tue Jun 30 01:46:54 2020: Errors in precompilation:
[E] Tue Jun 30 01:46:54 2020: error: unable to open output file '/usr/local/include/vc4cl-stdlib//VC4CLStdLib.h.pch': 'Permission denied'
1 error generated.

terminate called after throwing an instance of 'vc4c::CompilationError'
  what():  Pre-compilation: Error in precompilation: error: unable to open output file '/usr/local/include/vc4cl-stdlib//VC4CLStdLib.h.pch': 'Permission denied'
1 error generated.

[E] Tue Jun 30 01:46:54 2020: Received signal: SIGABRT
make[2]: *** [src/CMakeFiles/VC4C.dir/build.make:105: src/vc4c] Error 6
make[2]: *** Deleting file 'src/vc4c'
make[2]: Leaving directory '/home/satcom886/.cache/yay/vc4c-git/src/VC4C/build'
make[1]: *** [CMakeFiles/Makefile2:2037: src/CMakeFiles/VC4C.dir/all] Error 2
make[1]: Leaving directory '/home/satcom886/.cache/yay/vc4c-git/src/VC4C/build'
make: *** [Makefile:180: all] Error 2
bronze51 commented 4 years ago

With khronos-ocl-icd (per https://github.com/doe300/VC4CL/issues/93#issuecomment-653734715) I think I made progress. I ran:

cmake -DBUILD_TESTING=OFF -DSPIRV_FRONTEND=OFF -DCLANG_FOUND=/usr/bin/clang -DVC4CL_STDLIB_PRECOMPILE=OFF ..

Perhaps VC4CL_STDLIB_PRECOMPILE is not needed in this build scenario as I got "Manually-specified variables were not used by the project: VC4CL_STDLIB_PRECOMPILE"

Next I ran make:

...
[ 98%] Built target GetGitCommit
Scanning dependencies of target VC4C
[ 98%] Building CXX object src/CMakeFiles/VC4C.dir/main.cpp.o
[ 98%] Linking CXX executable vc4c
~/dev/VC4C/build/src/vc4c: error while loading shared libraries: ~/dev/VC4C/build/src/libVC4CC.so.1.2: unexpected reloc type 0x03
make[2]: *** [src/CMakeFiles/VC4C.dir/build.make:108: src/vc4c] Error 127
make[2]: *** Deleting file 'src/vc4c'
make[1]: *** [CMakeFiles/Makefile2:1965: src/CMakeFiles/VC4C.dir/all] Error 2
make: *** [Makefile:172: all] Error 2
bronze51 commented 4 years ago

I succeeded building VC4C using the AUR build procedures! (vc4c-git)

However, I had to turn off distcc to get around:

[ 82%] Built target VC4CC
Scanning dependencies of target VC4C
[ 84%] Built target qpu_emulator
[ 98%] Built target TestVC4C
[100%] Building CXX object src/CMakeFiles/VC4C.dir/main.cpp.o
distcc[22756] (main) CRITICAL! distcc seems to have invoked itself recursively!distcc[22756] (main) CRITICAL! distcc seems to have invoked itself recursively!
distcc[22755] ERROR: compile (null) on localhost failed with exit code 111
distcc[22754] (dcc_select_for_write) ERROR: IO timeout
distcc[22754] ERROR: timeout while connecting to 192.168.1.190:3632
distcc[22754] Warning: failed to distribute ~/dev/vc4c-git/src/VC4C/src/main.cpp to 192.168.1.190, running locally instead
distcc[22758] (main) CRITICAL! distcc seems to have invoked itself recursively!
distcc[22757] ERROR: compile (null) on localhost failed with exit code 111
distcc[22754] ERROR: compile ~/dev/vc4c-git/src/VC4C/src/main.cpp on localhost failed with exit code 111
make[2]: *** [src/CMakeFiles/VC4C.dir/build.make:83: src/CMakeFiles/VC4C.dir/main.cpp.o] Error 111
make[1]: *** [CMakeFiles/Makefile2:2042: src/CMakeFiles/VC4C.dir/all] Error 2
make: *** [Makefile:183: all] Error 2
==> ERROR: A failure occurred in package().
Aborting...

cmake (modified by me and copied from the Arch build files) is invoked with CLANG_FOUND, but it seems to be ignored at this point of the process?

cmake "$srcdir/VC4C" -DCMAKE_BUILD_TYPE=Release -DMULTI_THREADED=true -DCLANG_FOUND=/usr/bin/clang -DVC4CL_STDLIB_DIR=/usr/local/include/vc4cl-stdlib -DVC4CL_STDLIB_PRECOMPILE=OFF -Wno-dev -DBUILD_TESTING=OFF -DSPIRV_FRONTEND=OFF