pfllo commented 9 years ago

Hi, when I try to install cltorch with "luarocks install cltorch", I get the following errors:

Linking CXX shared library libTHCl.dylib
Undefined symbols for architecture x86_64:
  "THClKernels* THClKernels::in<unsigned long long>(TensorInfo<unsigned long long>)", referenced from:
      void kernelLaunch_THClTensor_reduceAllPass1<unsigned long long>(THClState*, dim3&, dim3&, unsigned long, int, TensorInfo<unsigned long long> const&, long, float, HasOperator2 const*, HasOperator3 const*, CLWrapper*) in THClReduceAll.cpp.o
      void kernelLaunch_THClTensor_reduceAll<unsigned long long>(THClState*, dim3&, dim3&, unsigned long, int, TensorInfo<unsigned long long> const&, long, float, HasOperator2 const*, HasOperator3 const*, CLWrapper*) in THClReduceAll.cpp.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [lib/THCl/libTHCl.dylib] Error 1
make[1]: *** [lib/THCl/CMakeFiles/THCl.dir/all] Error 2
make: *** [all] Error 2

I'm using OS X 10.10.2 and Intel Iris Graphics 6100.

Can you tell me how to fix these errors?

Many thanks!

hughperkins commented 9 years ago

I'm not sure. I guess a .dylib is like an .so , right? So, it is linking the libThCl.dylib so, and it's trying to add in the THClReduceAll.cpp.o object, which is part of libTHCl.dylib, and then it says it's missing ReduceAllPass1 and ReduceAll methods. These are actually templated methods, in the header file THClReduceAll.h, I think. I cant think why it woud work for linux but not mac. Maybe I should try building my code with clang perhaps?

hughperkins commented 9 years ago

Well, I installed clang-3.5, and built cltorch like this (on ubuntu), and it produced a whole bunch of new warnings, that I havent seen before, but it did build:

mkdir build-clang
cd build-clang
export CC=/usr/bin/clang-3.5
export CXX=/usr/bin/clang++-3.5
ccmake ..
make -j 4

Could be something to do wit hthe llvm linker though. Maybe I should try that.

hughperkins commented 9 years ago

What happens if you paste the following at the bottom of lib/THCl/THClReduceAll.cpp, and rebuild? :

template<>
void kernelLaunch_THClTensor_reduceAllPass1<unsigned long long>(
                     THClState* state,
                     dim3 &grid, dim3 &block, size_t smemSize,
                     int ADims,
                     const TensorInfo<unsigned long long> & in,
//                     CLWrapper *in_data,
                     long totalElements,
                     float init,
                     const HasOperator2 *modifyOp,
                     const HasOperator3 *reduceOp,
                     CLWrapper* scratch
    );
template<>
void kernelLaunch_THClTensor_reduceAll(
                     THClState* state,
                     dim3 &grid, dim3 &block, size_t smemSize,
                     int ADims,
                     const TensorInfo<unsigned long long> &in,
//                     CLWrapper *in_data,
                     long totalElements,
                     float init,
                     const HasOperator2 *modifyOp,
                     const HasOperator3 *reduceOp,
                     CLWrapper* devOut
    );

hughperkins commented 9 years ago

Hmmm, those angle brackets in my last comment probably shouldnt be there. Probalby something more like this:

template
void kernelLaunch_THClTensor_reduceAllPass1<unsigned long long>(
                     THClState* state,
                     dim3 &grid, dim3 &block, size_t smemSize,
                     int ADims,
                     const TensorInfo<unsigned long long> & in,
//                     CLWrapper *in_data,
                     long totalElements,
                     float init,
                     const HasOperator2 *modifyOp,
                     const HasOperator3 *reduceOp,
                     CLWrapper* scratch
    );
template
void kernelLaunch_THClTensor_reduceAll<unsigned long long>(
                     THClState* state,
                     dim3 &grid, dim3 &block, size_t smemSize,
                     int ADims,
                     const TensorInfo<unsigned long long> &in,
//                     CLWrapper *in_data,
                     long totalElements,
                     float init,
                     const HasOperator2 *modifyOp,
                     const HasOperator3 *reduceOp,
                     CLWrapper* devOut
    );

pfllo commented 9 years ago

After pasting the code above (without angle brackets), it generated the following errors:

16 warnings generated.
Linking CXX shared library libTHCl.dylib
Undefined symbols for architecture x86_64:
  "void kernelLaunch_THClTensor_reduceAll<unsigned long long>(THClState*, dim3&, dim3&, unsigned long, int, TensorInfo<unsigned long long> const&, long, float, HasOperator2 const*, HasOperator3 const*, CLWrapper*)", referenced from:
      void callReduceAll<unsigned long long>(THClState*, int, TensorInfo<unsigned long long> const&, long, float, HasOperator2 const*, HasOperator3 const*, CLWrapper*) in THClReduceAll.cpp.o
  "void kernelLaunch_THClTensor_reduceAllPass1<unsigned long long>(THClState*, dim3&, dim3&, unsigned long, int, TensorInfo<unsigned long long> const&, long, float, HasOperator2 const*, HasOperator3 const*, CLWrapper*)", referenced from:
      void callReduceAll<unsigned long long>(THClState*, int, TensorInfo<unsigned long long> const&, long, float, HasOperator2 const*, HasOperator3 const*, CLWrapper*) in THClReduceAll.cpp.o
  "void kernelLaunch_THClTensor_reduceAllPass2<unsigned long long>(THClState*, dim3&, dim3&, unsigned long, int, float, HasOperator3 const*, CLWrapper*, CLWrapper*)", referenced from:
      void callReduceAll<unsigned long long>(THClState*, int, TensorInfo<unsigned long long> const&, long, float, HasOperator2 const*, HasOperator3 const*, CLWrapper*) in THClReduceAll.cpp.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [lib/THCl/libTHCl.dylib] Error 1
make[1]: *** [lib/THCl/CMakeFiles/THCl.dir/all] Error 2
make: *** [all] Error 2

If I use angle brackets, it says this (3 warnings removed):

13 warnings generated.
Linking CXX shared library libTHCl.dylib
Undefined symbols for architecture x86_64:
  "void kernelLaunch_THClTensor_reduceAll<unsigned long long>(THClState*, dim3&, dim3&, unsigned long, int, TensorInfo<unsigned long long> const&, long, float, HasOperator2 const*, HasOperator3 const*, CLWrapper*)", referenced from:
      void callReduceAll<unsigned long long>(THClState*, int, TensorInfo<unsigned long long> const&, long, float, HasOperator2 const*, HasOperator3 const*, CLWrapper*) in THClReduceAll.cpp.o
  "void kernelLaunch_THClTensor_reduceAllPass1<unsigned long long>(THClState*, dim3&, dim3&, unsigned long, int, TensorInfo<unsigned long long> const&, long, float, HasOperator2 const*, HasOperator3 const*, CLWrapper*)", referenced from:
      void callReduceAll<unsigned long long>(THClState*, int, TensorInfo<unsigned long long> const&, long, float, HasOperator2 const*, HasOperator3 const*, CLWrapper*) in THClReduceAll.cpp.o
  "void kernelLaunch_THClTensor_reduceAllPass2<unsigned long long>(THClState*, dim3&, dim3&, unsigned long, int, float, HasOperator3 const*, CLWrapper*, CLWrapper*)", referenced from:
      void callReduceAll<unsigned long long>(THClState*, int, TensorInfo<unsigned long long> const&, long, float, HasOperator2 const*, HasOperator3 const*, CLWrapper*) in THClReduceAll.cpp.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [lib/THCl/libTHCl.dylib] Error 1
make[1]: *** [lib/THCl/CMakeFiles/THCl.dir/all] Error 2
make: *** [all] Error 2

hughperkins commented 9 years ago

Hmmm, thanks. Will ponder a bit.

hughperkins commented 9 years ago

Can you run llvm-nm on your .obj file pleaes, and paste the results here? Something like:

user@pear:~/git/cltorch/build-clang$ llvm-nm-3.5 lib/THCl/CMakeFiles/THCl.dir/THClReduceAll.cpp.o  | grep ReduceAll
0000000000000040 t _GLOBAL__sub_I_THClReduceAll.cpp
0000000000000000 W _Z13callReduceAllIjEvP9THClStateiRK10TensorInfoIT_ElfPK12HasOperator2PK12HasOperator3P9CLWrapper
0000000000000000 W _Z13callReduceAllIyEvP9THClStateiRK10TensorInfoIT_ElfPK12HasOperator2PK12HasOperator3P9CLWrapper
0000000000000000 W _Z21getReduceAllBlockSizeP9THClState
00000000000003b0 T _Z31THClReduceAll_getKernelTemplatev
user@pear:~/git/cltorch/build-clang$

Also, if you have nm available, can you do the same thing, but using nm -c instead of llvm-nm?

Ambext commented 9 years ago

Hello, sorry to add issues to issues, but I too have some problem building cltorch, either from luarocks or from your github repository FYI I am running all this on a 27' late 2014 Retina iMac, 4Ghz i7, 16Gb Ram, AMD Radeon R9 295X 4Gb, under Yosemite 10.10.4 (didn't work better for 10.10.3)

With luarocks I can this at the last lines [ 35%] Completed 'EasyCL-external' [ 35%] Built target EasyCL-external make: *\ [all] Error 2

I can send the full log is useful

With a direct build (ie doing git clone --recursive https://github.com/hughperkins/cltorch.git cd cltorch mkdir build cd build cmake .. make )

I get this at the make step

make[2]: * [clBLAS/stamp/clBLAS-external-configure] Error 1 make[1]: * [CMakeFiles/clBLAS-external.dir/all] Error 2 make: *\ [all] Error 2

same, happy to produce the complete log file if useful.

I have been investigating a little bit but with my very limited understanding I haven't figured out much so far.

Thanks heaps for doing all this !

Ambroise.

pfllo commented 9 years ago

Sorry to reply late. I just had some trouble installing llvm due to my network (already fixed).

And sorry again, I don't have the directory "lib/THCl/CMakeFiles" after building the project. Probably because the building process terminated before creating this directory. (not sure)

hughperkins commented 9 years ago

@luo123n. Hmmm, perhaps you can send the reulst of an ls -R? Maybe even zip the whole build directory up, and upload it to somewhere accessible?

hughperkins commented 9 years ago

@Ambext : yes, please send logfiles. Also, try running the bulid command a couple of times, see if it somehow 'goes through' the second time . Sometimes the dpeendnecies dont build in quite hte right order, but hte second pass, they work ok.

Ambext commented 9 years ago

I have tried 10+ times before bothering you believe me ;-)

You will find the two log files attached.

thanks again,

Ambroise.

On 2 Jul 2015, at 20:07, Hugh Perkins notifications@github.com wrote:

@Ambext https://github.com/Ambext : yes, please send logfiles. Also, try running the bulid command a couple of times, see if it somehow 'goes through' the second time . Sometimes the dpeendnecies dont build in quite hte right order, but hte second pass, they work ok.

— Reply to this email directly or view it on GitHub https://github.com/hughperkins/cltorch/issues/3#issuecomment-117988484.

Last login: Thu Jul 2 20:31:10 on ttys000 Exowide:~ mnemonis$ git clone --recursive https://github.com/hughperkins/cltorch.git Cloning into 'cltorch'... remote: Counting objects: 2456, done. remote: Compressing objects: 100% (120/120), done. remote: Total 2456 (delta 131), reused 74 (delta 74), pack-reused 2262 Receiving objects: 100% (2456/2456), 528.07 KiB | 289.00 KiB/s, done. Resolving deltas: 100% (1580/1580), done. Checking connectivity... done. Submodule 'EasyCL' (https://github.com/hughperkins/EasyCL.git) registered for path 'EasyCL' Submodule 'clBLAS' (https://github.com/hughperkins/clBLAS) registered for path 'clMathLibraries/clBLAS' Cloning into 'EasyCL'... remote: Counting objects: 1254, done. remote: Compressing objects: 100% (66/66), done. remote: Total 1254 (delta 35), reused 0 (delta 0), pack-reused 1185 Receiving objects: 100% (1254/1254), 706.61 KiB | 350.00 KiB/s, done. Resolving deltas: 100% (796/796), done. Checking connectivity... done. Submodule path 'EasyCL': checked out 'a03261662d263f14ce8e8e6f4dce1f649b400bd0' Cloning into 'clMathLibraries/clBLAS'... remote: Counting objects: 1978, done. remote: Compressing objects: 100% (25/25), done. remote: Total 1978 (delta 9), reused 0 (delta 0), pack-reused 1953 Receiving objects: 100% (1978/1978), 1.84 MiB | 754.00 KiB/s, done. Resolving deltas: 100% (1224/1224), done. Checking connectivity... done. Submodule path 'clMathLibraries/clBLAS': checked out 'bdaf1f9ef3d97168fdaff4c3ef588feeae72b171' Exowide:~ mnemonis$ cd cltorch Exowide:cltorch mnemonis$ mkdir build Exowide:cltorch mnemonis$ cd build Exowide:build mnemonis$ cmkae .. -bash: cmkae: command not found Exowide:build mnemonis$ cmake .. -- The C compiler identification is AppleClang 6.1.0.6020053 -- The CXX compiler identification is AppleClang 6.1.0.6020053 -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found Torch7 in /Users/mnemonis/torch/install /Users/mnemonis/cltorch/build /Users/mnemonis/cltorch On branch master Your branch is up-to-date with 'origin/master'. nothing to commit, working directory clean Submodule path 'EasyCL': checked out 'a03261662d263f14ce8e8e6f4dce1f649b400bd0' Submodule path 'clMathLibraries/clBLAS': checked out 'bdaf1f9ef3d97168fdaff4c3ef588feeae72b171' CMAKE_INSTALL_PREFIX /Users/mnemonis/torch/install CMAKE_INSTALL_PREFIX /Users/mnemonis/torch/install CLBLAS_INCLUDE_DIRS /Users/mnemonis/torch/install/include DEEPCL_LIBRARIES /Users/mnemonis/torch/install/lib/libEasyCL.dylib git version 2.4.5 -- Configuring done -- Generating done -- Build files have been written to: /Users/mnemonis/cltorch/build Exowide:build mnemonis$ make Scanning dependencies of target easycl_delete_stamp [ 0%] Built target easycl_delete_stamp Scanning dependencies of target git_cltorch_rec Submodule path '../EasyCL': checked out 'a03261662d263f14ce8e8e6f4dce1f649b400bd0' Submodule path '../clMathLibraries/clBLAS': checked out 'bdaf1f9ef3d97168fdaff4c3ef588feeae72b171' [ 0%] Built target git_cltorch_rec Scanning dependencies of target EasyCL-external [ 1%] Creating directories for 'EasyCL-external' [ 3%] No download step for 'EasyCL-external' [ 5%] No patch step for 'EasyCL-external' [ 7%] No update step for 'EasyCL-external' [ 8%] Performing configure step for 'EasyCL-external' loading initial cache file /Users/mnemonis/cltorch/build/EasyCL/tmp/EasyCL-external-cache.cmake -- The C compiler identification is AppleClang 6.1.0.6020053 -- The CXX compiler identification is AppleClang 6.1.0.6020053 -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Configuring done CMake Warning (dev): Policy CMP0042 is not set: MACOSX_RPATH is enabled by default. Run "cmake --help-policy CMP0042" for policy details. Use the cmake_policy command to set the policy and suppress this warning.

MACOSX_RPATH is not specified for the following targets:

EasyCL

This warning is for project developers. Use -Wno-dev to suppress it.

-- Generating done -- Build files have been written to: /Users/mnemonis/cltorch/build/EasyCL/src/EasyCL-external-build [ 10%] Performing build step for 'EasyCL-external' Scanning dependencies of target CLEW [ 2%] Building C object CMakeFiles/CLEW.dir/thirdparty/clew/src/clew.c.o In file included from /Users/mnemonis/cltorch/EasyCL/thirdparty/clew/src/clew.c:9: /Users/mnemonis/cltorch/EasyCL/./clew.h:1931:51: warning: 'weak_import' attribute only applies to variables and functions [-Wignored-attributes] ...cl_int * /* errcoderet /) CL_API_SUFFIXVERSI... ^ /Users/mnemonis/cltorch/EasyCL/./clew.h:80:53: note: expanded from macro 'CL_API_SUFFIXVERSION_1_1'

define CL_API_SUFFIX__VERSION_1_1 CL_EXTENSION_WEAK_LINK