make error on OSX 10.12 / CUDA 8

[subzerofun]

Hey thanks for the code, unfortunately i get a compile error when i try to build the cuda_utils.cu:

make clean && make
find . -type f | xargs -n 5 touch
rm -f libcuda_utils.so
/usr/local/cuda/bin/nvcc -arch sm_35 -O3 -DNDEBUG --compiler-options '-fPIC' -o libcuda_utils.so --shared cuda_utils.cu -I/Users/david/torch/install/include/THC -I/Users/david/torch/install/include/TH -I/Users/david/torch/install/include -L/Users/david/torch/install/lib -Xlinker -rpath,/Users/david/torch/install/lib -lluaT -lTHC -lTH -lpng
Undefined symbols for architecture x86_64:
  "_luaL_checknumber", referenced from:
      matting_laplacian(lua_State*) in tmpxft_0001773d_00000000-16_cuda_utils.o
      smooth_local_affine(lua_State*) in tmpxft_0001773d_00000000-16_cuda_utils.o
  "_luaL_error", referenced from:
      checkCudaError(lua_State*) in tmpxft_0001773d_00000000-16_cuda_utils.o
      matting_laplacian(lua_State*) in tmpxft_0001773d_00000000-16_cuda_utils.o
      smooth_local_affine(lua_State*) in tmpxft_0001773d_00000000-16_cuda_utils.o
  "_luaL_openlib", referenced from:
      _luaopen_libcuda_utils in tmpxft_0001773d_00000000-16_cuda_utils.o
  "_lua_call", referenced from:
      getCutorchState(lua_State*) in tmpxft_0001773d_00000000-16_cuda_utils.o
      matting_laplacian(lua_State*) in tmpxft_0001773d_00000000-16_cuda_utils.o
      smooth_local_affine(lua_State*) in tmpxft_0001773d_00000000-16_cuda_utils.o
  "_lua_getfield", referenced from:
      getCutorchState(lua_State*) in tmpxft_0001773d_00000000-16_cuda_utils.o
      matting_laplacian(lua_State*) in tmpxft_0001773d_00000000-16_cuda_utils.o
      smooth_local_affine(lua_State*) in tmpxft_0001773d_00000000-16_cuda_utils.o
  "_lua_settop", referenced from:
      getCutorchState(lua_State*) in tmpxft_0001773d_00000000-16_cuda_utils.o
      matting_laplacian(lua_State*) in tmpxft_0001773d_00000000-16_cuda_utils.o
      smooth_local_affine(lua_State*) in tmpxft_0001773d_00000000-16_cuda_utils.o
  "_lua_touserdata", referenced from:
      getCutorchState(lua_State*) in tmpxft_0001773d_00000000-16_cuda_utils.o
      matting_laplacian(lua_State*) in tmpxft_0001773d_00000000-16_cuda_utils.o
      smooth_local_affine(lua_State*) in tmpxft_0001773d_00000000-16_cuda_utils.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [libcuda_utils.so] Error 1

I've adjusted the links in the makefile and checked my torch install – it should be working. Am i missing something or does OSX need some specific compiler arguments?

[luanfujun]

Could you try xcode-select --install? Also could you check whether "require cutorch" in torch works normally? The code has only been tested on ubuntu currently.

[subzerofun]

Thanks for replying! Have been trying to fix the makefile arguments for 2 hours now...

The Command Line Tools are installed - for XCode 8.2. I just discovered that when i add -llua at the end of LDFLAGS_NVCC the errors are reduced to one (compared to the wall of errors above).

make
/usr/local/cuda/bin/nvcc -arch sm_35 -O3 -DNDEBUG --compiler-options '-fPIC' -o libcuda_utils.so --shared cuda_utils.cu -I/Users/david/torch/install/include/THC -I/Users/david/torch/install/include/TH -I/Users/david/torch/install/include -L/Users/david/torch/install/lib -Xlinker -rpath,/Users/david/torch/install/lib -lluaT -lTHC -lTH -lpng -llua
Undefined symbols for architecture x86_64:
  "_lua_call", referenced from:
      getCutorchState(lua_State*) in tmpxft_00011308_00000000-16_cuda_utils.o
      matting_laplacian(lua_State*) in tmpxft_00011308_00000000-16_cuda_utils.o
      smooth_local_affine(lua_State*) in tmpxft_00011308_00000000-16_cuda_utils.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [libcuda_utils.so] Error 1

I tried installing lua 5.1 via homebrew and also 5.2 – then switched between the two to see if it changes something. I also tried to completely uninstall both – but the outcome stayed the same. Of course i've added the /usr/local/lib + include to the makefile flags. But as that didn't make a difference i let the compiler load all lua libs from my torch install folder (default setting from your makefile).

Somehow the compiler doesn't find the necessary libraries for the 3 functions above. Will try to reinstall cutorch – but it don't think it will help. Have already reinstalled it 2 hours ago (without errors).

Do you have any idea what else i could try? At least i could reduce the errors to only one function – the _lua_call. If i get past that, the file should successfully compile.

Do i have to manually add another flag? The -llua flag helped a lot, maybe there is just one more missing to fix the _lua_call error. Do you know from which library _lua_call gets referenced?

[subzerofun]

Ah sorry – require cutorch works. I've been using it for a few other projects (neural-style and super-resolution repos) without problems.

th

  ______             __   |  Torch7
 /_  __/__  ________/ /   |  Scientific computing for Lua.
  / / / _ \/ __/ __/ _ \  |  Type ? for help
 /_/  \___/_/  \__/_//_/  |  https://github.com/torch
                          |  http://torch.ch

th> require cutorch
..>

[luanfujun]

#include "lua.h"
#include "lualib.h"
#include "lauxlib.h"

Could you check where the corresponding libs are located in OSX? I suspect there is some missing linking flag for lualib.h or lauxlib.h too in Mac, similar to the one you added -llua.

[subzerofun]

lua.h, lualib.h and lauxlib.h are located here: ~/torch/install/include

As well as here:

~/torch/exe/lua-5.1
~/torch/exe/lua-5.1-rc
~/torch/exe/lua-5.2
~/torch/exe/lua-5.3
~/torch/exe/luajit-2.0
~/torch/exe/luajit-2.1
~/torch/exe/luarocks

And from my brew install: /usr/local/lib

But isn't only the first folder is relevant? Since the other ones aren't listed in the makefile.

---

Could you please take a look at the way Facebook integrated the function getCutorchState here: https://github.com/facebook/fbcunn/blob/master/src/Utils.cpp

Is the comment

// Unfortunately cutorch lua headers aren't exported, so we have to // copy this. This is a copy from cunn.

maybe revealing a problem with cutorch?

I'm just searching for every clue – if other people had a similar problem with the functions from my error.

But so far i didn't find anything useful...

[subzerofun]

It looks like all .h files from extern "C" { ... } don't get loaded – if i uncomment them nothing changes, i get exactly the same error.

[subzerofun]

🎉🎉🎉 SUCCESS after 4 hours :-)  🎉🎉🎉

The correct OSX flags for LDFLAGS_NVCC are: LDFLAGS_NVCC=-L$(PREFIX)/lib -Xlinker -rpath,$(PREFIX)/lib -lluaT -lTHC -lTH -lpng -llua -lluajit -lm -ldl

I didn't check each of the added flags: -lluajit -lm -ldl – but i guess it doesn't hurt to add all of them.

Hopefully i can FINALLY try out your project now.

How many hours one has to waste until a simple error is fixed... reminds me of the first time i tried to compile Caffe. Only took me two days to fix it :-)

[luanfujun]

thanks @subzerofun! I will udpate it to the makefile for OSX users.

btw, extern "C" { ... } is required otherwise lua code can't load it.

[mosspassion]

Just to be clear to OS X users (I'm on OS X 10.11.6, Cuda 8):

You have to change /deep-photo-styletransfer/makefile: specifically where PREFIX is referring to (where torch is installed), where NVCC_PREFIX is referring to (where cuda is installed), and adding the -lluajit to LDFLAGS_NVCC as @subzerofun has contributed. Here is an example:

PREFIX=/Users/nicholasdavis/torch/install # my torch is installed at ~/torch NVCC_PREFIX=/usr/local/cuda/bin # my cuda was not at /cuda-7.5 so I had to change this too CFLAGS=-I$(PREFIX)/include/THC -I$(PREFIX)/include/TH -I$(PREFIX)/include # for OSX users add "-lluajit" to LDFLAGS_NVCC, thanks to @subzerofun LDFLAGS_NVCC=-L$(PREFIX)/lib -Xlinker -rpath,$(PREFIX)/lib -lluaT -lTHC -lTH -lpng -lluajit #here is where I added the -lluajit

<...>