Cannot build: dyld: Library not loaded: @rpath/libcudart.7.0.dylib on Mac OS X 10.11

tamasgal commented 9 years ago

I tried setting DYLD_LIBRARY_PATH and DYLD_FALLBACK_LIBRARY_PATH like suggested in other (older) CUDA projects, however, it does not work.

I installed CUDA via homebrew (cask) and also tried with the newest drivers and toolkit directly from NVIDIA (v7.5). Any ideas why the library is not getting loaded? It's definitely there…

Resolving dependencies...
Configuring cuda-0.6.6.2...
Building cuda-0.6.6.2...
Failed to install cuda-0.6.6.2
Build log ( /Users/tamasgal/.cabal/logs/cuda-0.6.6.2.log ):
[1 of 1] Compiling Main             ( /var/folders/n2/fpv312vd5xg91ncw_s5f1m5m0000gn/T/cuda-0.6.6.2-1871/cuda-0.6.6.2/dist/setup/setup.hs, /var/folders/n2/fpv312vd5xg91ncw_s5f1m5m0000gn/T/cuda-0.6.6.2-1871/cuda-0.6.6.2/dist/setup/Main.o )
Linking /var/folders/n2/fpv312vd5xg91ncw_s5f1m5m0000gn/T/cuda-0.6.6.2-1871/cuda-0.6.6.2/dist/setup/setup ...
Configuring cuda-0.6.6.2...
checking for gcc... /usr/bin/gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether /usr/bin/gcc accepts -g... yes
checking for /usr/bin/gcc option to accept ISO C89... none needed
checking build system type... x86_64-apple-darwin15.0.0
checking host system type... x86_64-apple-darwin15.0.0
checking target system type... x86_64-apple-darwin15.0.0
checking for nvcc... /usr/local/cuda/bin/nvcc
checking ghc version... 7.10.1
checking ghc architecture... x86_64
checking for Apple Blocks extension... no
checking how to run the C preprocessor... /usr/bin/gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking cuda.h usability... yes
checking cuda.h presence... yes
checking for cuda.h... yes
checking cuda_runtime_api.h usability... yes
checking cuda_runtime_api.h presence... yes
checking for cuda_runtime_api.h... yes
checking for library containing cuDriverGetVersion... -lcuda
checking for library containing cudaRuntimeGetVersion... -lcudart
configure: creating ./config.status
config.status: creating cuda.buildinfo
Building cuda-0.6.6.2...
Preprocessing library cuda-0.6.6.2...
dyld: Library not loaded: @rpath/libcudart.7.0.dylib
  Referenced from: /private/var/folders/n2/fpv312vd5xg91ncw_s5f1m5m0000gn/T/cuda-0.6.6.2-1871/cuda-0.6.6.2/dist/build/Foreign/CUDA/Internal/Offsets_hsc_make
  Reason: image not found
running dist/build/Foreign/CUDA/Internal/Offsets_hsc_make failed (exit code -5)
command was: dist/build/Foreign/CUDA/Internal/Offsets_hsc_make  >dist/build/Foreign/CUDA/Internal/Offsets.hs
cabal: Error: some packages failed to install:
cuda-0.6.6.2 failed during the building phase. The exception was:
ExitFailure 1

tamasgal commented 9 years ago

I updated cabal (forgot to do so before…), however, I got almost the same error:

Resolving dependencies...
Configuring cuda-0.6.7.0...
Building cuda-0.6.7.0...
Failed to install cuda-0.6.7.0
Build log ( /Users/tamasgal/.cabal/logs/cuda-0.6.7.0.log ):
[1 of 1] Compiling Main             ( /var/folders/n2/fpv312vd5xg91ncw_s5f1m5m0000gn/T/cuda-0.6.7.0-3747/cuda-0.6.7.0/dist/setup/setup.hs, /var/folders/n2/fpv312vd5xg91ncw_s5f1m5m0000gn/T/cuda-0.6.7.0-3747/cuda-0.6.7.0/dist/setup/Main.o )
Linking /var/folders/n2/fpv312vd5xg91ncw_s5f1m5m0000gn/T/cuda-0.6.7.0-3747/cuda-0.6.7.0/dist/setup/setup ...
Configuring cuda-0.6.7.0...
Found CUDA toolkit at: /usr/local/cuda
Storing parameters to cuda.buildinfo.generated
Using build information from 'cuda.buildinfo.generated'.
Provide a 'cuda.buildinfo' file to override this behaviour.
Using build information from 'cuda.buildinfo.generated'.
Provide a 'cuda.buildinfo' file to override this behaviour.
Building cuda-0.6.7.0...
Preprocessing library cuda-0.6.7.0...
[ 1 of 27] Compiling Foreign.CUDA.Internal.C2HS ( Foreign/CUDA/Internal/C2HS.hs, dist/build/Foreign/CUDA/Internal/C2HS.o )
[ 2 of 27] Compiling Foreign.CUDA.Driver.Error ( dist/build/Foreign/CUDA/Driver/Error.hs, dist/build/Foreign/CUDA/Driver/Error.o )

Foreign/CUDA/Driver/Error.chs:29:1: Warning:
    The import of ‘Foreign.Ptr’ is redundant
      except perhaps to import instances from ‘Foreign.Ptr’
    To import instances alone, use: import Foreign.Ptr()
[ 3 of 27] Compiling Foreign.CUDA.Driver.Utils ( dist/build/Foreign/CUDA/Driver/Utils.hs, dist/build/Foreign/CUDA/Driver/Utils.o )
[ 4 of 27] Compiling Foreign.CUDA.Runtime.Error ( dist/build/Foreign/CUDA/Runtime/Error.hs, dist/build/Foreign/CUDA/Runtime/Error.o )

Foreign/CUDA/Runtime/Error.chs:34:1: Warning:
    The import of ‘Foreign.Ptr’ is redundant
      except perhaps to import instances from ‘Foreign.Ptr’
    To import instances alone, use: import Foreign.Ptr()

Foreign/CUDA/Runtime/Error.chs:37:1: Warning:
    The import of ‘System.IO.Unsafe’ is redundant
      except perhaps to import instances from ‘System.IO.Unsafe’
    To import instances alone, use: import System.IO.Unsafe()
[ 5 of 27] Compiling Foreign.CUDA.Runtime.Utils ( dist/build/Foreign/CUDA/Runtime/Utils.hs, dist/build/Foreign/CUDA/Runtime/Utils.o )
[ 6 of 27] Compiling Foreign.CUDA.Analysis.Device ( dist/build/Foreign/CUDA/Analysis/Device.hs, dist/build/Foreign/CUDA/Analysis/Device.o )
[ 7 of 27] Compiling Foreign.CUDA.Analysis.Occupancy ( Foreign/CUDA/Analysis/Occupancy.hs, dist/build/Foreign/CUDA/Analysis/Occupancy.o )
[ 8 of 27] Compiling Foreign.CUDA.Runtime.Device ( dist/build/Foreign/CUDA/Runtime/Device.hs, dist/build/Foreign/CUDA/Runtime/Device.o )

Foreign/CUDA/Runtime/Device.chs:33:1: Warning:
    The qualified import of ‘Foreign.ForeignPtr’ is redundant
      except perhaps to import instances from ‘Foreign.ForeignPtr’
    To import instances alone, use: import Foreign.ForeignPtr()
[ 9 of 27] Compiling Foreign.CUDA.Driver.Device ( dist/build/Foreign/CUDA/Driver/Device.hs, dist/build/Foreign/CUDA/Driver/Device.o )

Foreign/CUDA/Driver/Device.chs:25:1: Warning:
    The qualified import of ‘Foreign.ForeignPtr’ is redundant
      except perhaps to import instances from ‘Foreign.ForeignPtr’
    To import instances alone, use: import Foreign.ForeignPtr()
[10 of 27] Compiling Foreign.CUDA.Driver.Context ( dist/build/Foreign/CUDA/Driver/Context.hs, dist/build/Foreign/CUDA/Driver/Context.o )
[11 of 27] Compiling Foreign.CUDA.Analysis ( Foreign/CUDA/Analysis.hs, dist/build/Foreign/CUDA/Analysis.o )
[12 of 27] Compiling Foreign.CUDA.Types ( dist/build/Foreign/CUDA/Types.hs, dist/build/Foreign/CUDA/Types.o )
[13 of 27] Compiling Foreign.CUDA.Runtime.Event ( dist/build/Foreign/CUDA/Runtime/Event.hs, dist/build/Foreign/CUDA/Runtime/Event.o )
[14 of 27] Compiling Foreign.CUDA.Runtime.Stream ( dist/build/Foreign/CUDA/Runtime/Stream.hs, dist/build/Foreign/CUDA/Runtime/Stream.o )
[15 of 27] Compiling Foreign.CUDA.Runtime.Exec ( dist/build/Foreign/CUDA/Runtime/Exec.hs, dist/build/Foreign/CUDA/Runtime/Exec.o )

Foreign/CUDA/Runtime/Exec.chs:23:1: Warning:
    The qualified import of ‘Foreign.ForeignPtr’ is redundant
      except perhaps to import instances from ‘Foreign.ForeignPtr’
    To import instances alone, use: import Foreign.ForeignPtr()
[16 of 27] Compiling Foreign.CUDA.Driver.Event ( dist/build/Foreign/CUDA/Driver/Event.hs, dist/build/Foreign/CUDA/Driver/Event.o )
[17 of 27] Compiling Foreign.CUDA.Driver.Stream ( dist/build/Foreign/CUDA/Driver/Stream.hs, dist/build/Foreign/CUDA/Driver/Stream.o )
[18 of 27] Compiling Foreign.CUDA.Driver.Exec ( dist/build/Foreign/CUDA/Driver/Exec.hs, dist/build/Foreign/CUDA/Driver/Exec.o )
[19 of 27] Compiling Foreign.CUDA.Ptr ( Foreign/CUDA/Ptr.hs, dist/build/Foreign/CUDA/Ptr.o )
[20 of 27] Compiling Foreign.CUDA.Runtime.Marshal ( dist/build/Foreign/CUDA/Runtime/Marshal.hs, dist/build/Foreign/CUDA/Runtime/Marshal.o )
[21 of 27] Compiling Foreign.CUDA.Runtime.Texture ( dist/build/Foreign/CUDA/Runtime/Texture.hs, dist/build/Foreign/CUDA/Runtime/Texture.o )

Foreign/CUDA/Runtime/Texture.chs:21:1: Warning:
    The qualified import of ‘Foreign.ForeignPtr’ is redundant
      except perhaps to import instances from ‘Foreign.ForeignPtr’
    To import instances alone, use: import Foreign.ForeignPtr()
[22 of 27] Compiling Foreign.CUDA.Driver.Marshal ( dist/build/Foreign/CUDA/Driver/Marshal.hs, dist/build/Foreign/CUDA/Driver/Marshal.o )
[23 of 27] Compiling Foreign.CUDA.Driver.Texture ( dist/build/Foreign/CUDA/Driver/Texture.hs, dist/build/Foreign/CUDA/Driver/Texture.o )
[24 of 27] Compiling Foreign.CUDA.Driver.Module ( dist/build/Foreign/CUDA/Driver/Module.hs, dist/build/Foreign/CUDA/Driver/Module.o )
[25 of 27] Compiling Foreign.CUDA.Driver ( Foreign/CUDA/Driver.hs, dist/build/Foreign/CUDA/Driver.o )
[26 of 27] Compiling Foreign.CUDA.Runtime ( Foreign/CUDA/Runtime.hs, dist/build/Foreign/CUDA/Runtime.o )
[27 of 27] Compiling Foreign.CUDA     ( Foreign/CUDA.hs, dist/build/Foreign/CUDA.o )
ld: file not found: @rpath/CUDA.framework/Versions/A/CUDA for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
cabal: Error: some packages failed to install:
cuda-0.6.7.0 failed during the building phase. The exception was:
ExitFailure 1

Which I could finally fix by changing the libcuda.dylib dependency using install_name_tool:

sudo install_name_tool -change @rpath/CUDA.framework/Versions/A/CUDA /Library/Frameworks/CUDA.framework/CUDA /usr/local/cuda/lib/libcuda.dylib

tamasgal commented 9 years ago

(sorry I accidentally clicked the close/open)

Still no luck:

λ :m +Foreign.CUDA
Prelude Foreign.CUDA
λ props 0
can't load .so/.DLL for: /Users/tamasgal/.cabal/lib/x86_64-osx-ghc-7.10.1/cuda_5NEysEVDNVg83Ls8aYPaNt/libHScuda-0.6.7.0-5NEysEVDNVg83Ls8aYPaNt-ghc7.10.1.dylib (dlopen(/Users/tamasgal/.cabal/lib/x86_64-osx-ghc-7.10.1/cuda_5NEysEVDNVg83Ls8aYPaNt/libHScuda-0.6.7.0-5NEysEVDNVg83Ls8aYPaNt-ghc7.10.1.dylib, 5): Library not loaded: @rpath/libcudart.7.5.dylib
  Referenced from: /Users/tamasgal/.cabal/lib/x86_64-osx-ghc-7.10.1/cuda_5NEysEVDNVg83Ls8aYPaNt/libHScuda-0.6.7.0-5NEysEVDNVg83Ls8aYPaNt-ghc7.10.1.dylib
  Reason: image not found)

tmcdonell commented 9 years ago

I haven't tried with the version from homebrew before. I'm currently running CUDA 7.5 (plus older versions) on 10.10.5, installed via the NVIDIA installer.

Is the homebrew installer placing the CUDA.framework into /Library/Frameworks or somewhere else, and the libraries in (what looks like should be) /usr/local/cuda/lib/?

Also to clarify, the hack with install_name_tool didn't work?

Check env | grep DYLD_LIBRARY_PATH ?

tamasgal commented 9 years ago

Yea, all the files are where they should be and the paths are correct. The hack with install_name_tool worked, so that I could compile cabal install cuda. But I get the error when trying to use it actually…

However, it seems that the whole NVIDIA package is somehow not really working. I could compile the sample programs of the CUDA framework but when I run them, I get this error:

tamasgal@greybox ~/Desktop/samples/0_Simple/vectorAdd                                                            [16:10:18] 
> $ ./vectorAdd                                                                                                            
[Vector addition of 50000 elements]
Failed to allocate device vector A (error code CUDA driver version is insufficient for CUDA runtime version)!

tmcdonell commented 9 years ago

Is this with homebrew still, or with the NVIDIA installer?

Also I just noticed you are using the El Capitan beta. I haven't tried that yet. If the CUDA examples themselves don't work, I suspect that you'll need to wait for an update on NVIDIA's end.

tmcdonell commented 9 years ago

@tamasgal I pushed a change and now [executables] run without needing to specify DYLD_LIBRARY_PATH (still having issues in ghci though). This might make some progress for you, at least until the insufficient driver version error. Can you give it a go?

trevor@nightfall ~/D/P/c/cuda (master)
> echo $DYLD_LIBRARY_PATH

trevor@nightfall ~/D/P/c/cuda (master)
> ./.cabal-sandbox/bin/nvidia-device-query
CUDA device query (Driver API, statically linked)
CUDA driver version 7.5
Detected 1 CUDA capable device
<snip>

tamasgal commented 9 years ago

Sorry for the late answer and many thanks for you efforts. First I took the homebrew version, then I tried it with the newest NVIDIA installer.

Unfortunately I don't have access to that machine right now and my own computer (MacBook Pro) has no NVIDIA video card installed. I'll give it a go in the next few days, but I think we can close the discussion since the major problem is obviously a not (yet) supported beta OS.

tmcdonell commented 9 years ago

Sure, let's look at this again once El Capitan is out and supported by NVIDIA. Thanks for your help so far!

tmcdonell / cuda

Cannot build: dyld: Library not loaded: @rpath/libcudart.7.0.dylib on Mac OS X 10.11 #32