search

uncomplicate / neanderthal

Fast Clojure Matrix Library
http://neanderthal.uncomplicate.org
Eclipse Public License 1.0
1.06k stars 56 forks source link

java.lang.UnsatisfiedLinkError on Mac OS X #15

Closed fonghou closed 8 years ago

fonghou commented 8 years ago

Hello,

First, sorry about the long post.

Here are my lein project.clj and a sample namespace. When loading it from lein repl, got a stacktrace list below.

(defproject matrix "0.1.0-SNAPSHOT" :dependencies [[org.clojure/clojure "1.8.0"] [com.taoensso/truss "1.2.0"] [uncomplicate/clojurecl "0.6.4"] [uncomplicate/fluokitten "0.5.0"] [uncomplicate/neanderthal "0.6.2"]]

:profiles {:dev {:dependencies [[criterium "0.4.4"]]} :uberjar {:aot :all}} )

(ns matrix.core (:require [uncomplicate.clojurecl.core :as cl] [uncomplicate.commons.core :refer [with-release]] [uncomplicate.neanderthal [core :refer [asum dot axpy! mv! mm! transfer! copy]] [native :refer [sv sge]] [opencl :refer [with-default-engine clv clge]]] [criterium.core :refer [quick-bench with-progress-reporting]]))

The interesting one is the nested cause:

Stack trace from the attempt to load the library as a resource: java.lang.UnsatisfiedLinkError: /private/var/folders/xm/vkg11d9n74ngq3zsbjlms04w0000gn/T/libJOCLBlast_0_7_1-apple-x86_64.dylib: dlopen(/private/var/folders/xm/vkg11d9n74ngq3zsbjlms04w0000gn/T/libJOCLBlast_0_7_1-apple-x86_64.dylib, 1): Library not loaded: libclblast.dylib Referenced from: /private/var/folders/xm/vkg11d9n74ngq3zsbjlms04w0000gn/T/libJOCLBlast_0_7_1-apple-x86_64.dylib

ls -lR that folder /private/var/... shows required dylib are there, but layout looks strange. Not sure this is leinignen issue or not.

/p/v/f/x/v/T ❯❯❯ ls -lR JOCL -rw-r--r-- 1 houf staff 318908 May 25 10:50 libJOCLBlast_0_7_1-apple-x86_64.dylib -rw-r--r-- 1 houf staff 180064 May 23 19:34 libJOCL_2_0_0-apple-x86_64.dylib

JOCLBlast_0_7_1-apple-x86_64_dependents: total 0 drwxr-xr-x 3 houf staff 102 May 25 10:50 apple

JOCLBlast_0_7_1-apple-x86_64_dependents/apple: total 0 drwxr-xr-x 3 houf staff 102 May 25 10:50 x86_64

JOCLBlast_0_7_1-apple-x86_64_dependents/apple/x86_64: total 4272 -rw-r--r-- 1 houf staff 2184612 May 25 10:50 libclblast.dylib

============Full Stacktrace ============== CompilerException java.lang.UnsatisfiedLinkError: Error while loading native library "JOCLBlast_0_7_1-apple-x86_64" Operating system name: Mac OS X Architecture : x86_64 Architecture bit size: 64 ---(start of nested stack traces)--- Stack trace from the attempt to load the library as a file: java.lang.UnsatisfiedLinkError: no JOCLBlast_0_7_1-apple-x86_64 in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at org.jocl.LibUtils.loadLibrary(LibUtils.java:136) at org.jocl.blast.CLBlast.(CLBlast.java:53) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at clojure.lang.RT.classForName(RT.java:2168) at clojure.lang.RT.classForName(RT.java:2177) at clojure.lang.Compiler$HostExpr.maybeClass(Compiler.java:1030) at clojure.lang.Compiler.macroexpand1(Compiler.java:6807) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6854) at clojure.lang.Compiler.analyze(Compiler.java:6669) at clojure.lang.Compiler.analyze(Compiler.java:6625) at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6001) at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:6319) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6868) at clojure.lang.Compiler.analyze(Compiler.java:6669) at clojure.lang.Compiler.analyze(Compiler.java:6625) at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6001) at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5380) at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3972) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6866) at clojure.lang.Compiler.analyze(Compiler.java:6669) at clojure.lang.Compiler.eval(Compiler.java:6924) at clojure.lang.Compiler.load(Compiler.java:7379) at clojure.lang.RT.loadResourceScript(RT.java:372) at clojure.lang.RT.loadResourceScript(RT.java:363) at clojure.lang.RT.load(RT.java:453) at clojure.lang.RT.load(RT.java:419) at clojure.core$load$fn5677.invoke(core.clj:5893) at clojure.core$load.invokeStatic(core.clj:5892) at clojure.core$load.doInvoke(core.clj:5876) at clojure.lang.RestFn.invoke(RestFn.java:408) at clojure.core$load_one.invokeStatic(core.clj:5697) at clojure.core$load_one.invoke(core.clj:5692) at clojure.core$load_lib$fn5626.invoke(core.clj:5737) at clojure.core$load_lib.invokeStatic(core.clj:5736) at clojure.core$load_lib.doInvoke(core.clj:5717) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invokeStatic(core.clj:648) at clojure.core$load_libs.invokeStatic(core.clj:5778) at clojure.core$load_libs.doInvoke(core.clj:5758) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:648) at clojure.core$require.invokeStatic(core.clj:5796) at clojure.core$require.doInvoke(core.clj:5796) at clojure.lang.RestFn.invoke(RestFn.java:482) at uncomplicate.neanderthal.opencl$eval37781$loading5569auto__37782.invoke(opencl.clj:1) at uncomplicate.neanderthal.opencl$eval37781.invokeStatic(opencl.clj:1) at uncomplicate.neanderthal.opencl$eval37781.invoke(opencl.clj:1) at clojure.lang.Compiler.eval(Compiler.java:6927) at clojure.lang.Compiler.eval(Compiler.java:6916) at clojure.lang.Compiler.load(Compiler.java:7379) at clojure.lang.RT.loadResourceScript(RT.java:372) at clojure.lang.RT.loadResourceScript(RT.java:363) at clojure.lang.RT.load(RT.java:453) at clojure.lang.RT.load(RT.java:419) at clojure.core$load$fn__5677.invoke(core.clj:5893) at clojure.core$load.invokeStatic(core.clj:5892) at clojure.core$load.doInvoke(core.clj:5876) at clojure.lang.RestFn.invoke(RestFn.java:408) at clojure.core$load_one.invokeStatic(core.clj:5697) at clojure.core$load_one.invoke(core.clj:5692) at clojure.core$load_lib$fn5626.invoke(core.clj:5737) at clojure.core$load_lib.invokeStatic(core.clj:5736) at clojure.core$load_lib.doInvoke(core.clj:5717) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invokeStatic(core.clj:648) at clojure.core$load_libs.invokeStatic(core.clj:5778) at clojure.core$load_libs.doInvoke(core.clj:5758) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:648) at clojure.core$require.invokeStatic(core.clj:5796) at clojure.core$require.doInvoke(core.clj:5796) at clojure.lang.RestFn.invoke(RestFn.java:457) at matrix.core$eval20272$loading5569auto__20273.invoke(form-init3071421537802120931.clj:1) at matrix.core$eval20272.invokeStatic(form-init3071421537802120931.clj:1) at matrix.core$eval20272.invoke(form-init3071421537802120931.clj:1) at clojure.lang.Compiler.eval(Compiler.java:6927) at clojure.lang.Compiler.eval(Compiler.java:6916) at clojure.lang.Compiler.eval(Compiler.java:6890) at clojure.core$eval.invokeStatic(core.clj:3105) at clojure.core$eval.invoke(core.clj:3101) at clojure.main$repl$read_eval_print7408$fn7411.invoke(main.clj:240) at clojure.main$repl$read_eval_print7408.invoke(main.clj:240) at clojure.main$repl$fn7417.invoke(main.clj:258) at clojure.main$repl.invokeStatic(main.clj:258) at clojure.main$repl.doInvoke(main.clj:174) at clojure.lang.RestFn.invoke(RestFn.java:1523) at clojure.tools.nrepl.middleware.interruptible_eval$evaluate$fn__941.invoke(interruptible_eval.clj:87) at clojure.lang.AFn.applyToHelper(AFn.java:152) at clojure.lang.AFn.applyTo(AFn.java:144) at clojure.core$apply.invokeStatic(core.clj:646) at clojure.core$with_bindingsSTAR.invokeStatic(core.clj:1881) at clojure.core$with_bindingsSTAR.doInvoke(core.clj:1881) at clojure.lang.RestFn.invoke(RestFn.java:425) at clojure.tools.nrepl.middleware.interruptible_eval$evaluate.invokeStatic(interruptible_eval.clj:85) at clojure.tools.nrepl.middleware.interruptible_eval$evaluate.invoke(interruptible_eval.clj:55) at clojure.tools.nrepl.middleware.interruptible_eval$interruptible_eval$fn986$fn989.invoke(interruptible_eval.clj:222) at clojure.tools.nrepl.middleware.interruptible_eval$run_next$fn__981.invoke(interruptible_eval.clj:190) at clojure.lang.AFn.run(AFn.java:22) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Stack trace from the attempt to load the library as a resource: java.lang.UnsatisfiedLinkError: /private/var/folders/xm/vkg11d9n74ngq3zsbjlms04w0000gn/T/libJOCLBlast_0_7_1-apple-x86_64.dylib: dlopen(/private/var/folders/xm/vkg11d9n74ngq3zsbjlms04w0000gn/T/libJOCLBlast_0_7_1-apple-x86_64.dylib, 1): Library not loaded: libclblast.dylib Referenced from: /private/var/folders/xm/vkg11d9n74ngq3zsbjlms04w0000gn/T/libJOCLBlast_0_7_1-apple-x86_64.dylib Reason: image not found at java.lang.ClassLoader$NativeLibrary.load(Native Method) at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824) at java.lang.Runtime.load0(Runtime.java:809) at java.lang.System.load(System.java:1086) at org.jocl.LibUtils.loadLibraryResource(LibUtils.java:269) at org.jocl.LibUtils.loadLibrary(LibUtils.java:151) at org.jocl.blast.CLBlast.(CLBlast.java:53) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at clojure.lang.RT.classForName(RT.java:2168) at clojure.lang.RT.classForName(RT.java:2177) at clojure.lang.Compiler$HostExpr.maybeClass(Compiler.java:1030) at clojure.lang.Compiler.macroexpand1(Compiler.java:6807) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6854) at clojure.lang.Compiler.analyze(Compiler.java:6669) at clojure.lang.Compiler.analyze(Compiler.java:6625) at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6001) at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:6319) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6868) at clojure.lang.Compiler.analyze(Compiler.java:6669) at clojure.lang.Compiler.analyze(Compiler.java:6625) at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6001) at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5380) at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3972) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6866) at clojure.lang.Compiler.analyze(Compiler.java:6669) at clojure.lang.Compiler.eval(Compiler.java:6924) at clojure.lang.Compiler.load(Compiler.java:7379) at clojure.lang.RT.loadResourceScript(RT.java:372) at clojure.lang.RT.loadResourceScript(RT.java:363) at clojure.lang.RT.load(RT.java:453) at clojure.lang.RT.load(RT.java:419) at clojure.core$load$fn5677.invoke(core.clj:5893) at clojure.core$load.invokeStatic(core.clj:5892) at clojure.core$load.doInvoke(core.clj:5876) at clojure.lang.RestFn.invoke(RestFn.java:408) at clojure.core$load_one.invokeStatic(core.clj:5697) at clojure.core$load_one.invoke(core.clj:5692) at clojure.core$load_lib$fn5626.invoke(core.clj:5737) at clojure.core$load_lib.invokeStatic(core.clj:5736) at clojure.core$load_lib.doInvoke(core.clj:5717) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invokeStatic(core.clj:648) at clojure.core$load_libs.invokeStatic(core.clj:5778) at clojure.core$load_libs.doInvoke(core.clj:5758) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:648) at clojure.core$require.invokeStatic(core.clj:5796) at clojure.core$require.doInvoke(core.clj:5796) at clojure.lang.RestFn.invoke(RestFn.java:482) at uncomplicate.neanderthal.opencl$eval37781$loading5569auto__37782.invoke(opencl.clj:1) at uncomplicate.neanderthal.opencl$eval37781.invokeStatic(opencl.clj:1) at uncomplicate.neanderthal.opencl$eval37781.invoke(opencl.clj:1) at clojure.lang.Compiler.eval(Compiler.java:6927) at clojure.lang.Compiler.eval(Compiler.java:6916) at clojure.lang.Compiler.load(Compiler.java:7379) at clojure.lang.RT.loadResourceScript(RT.java:372) at clojure.lang.RT.loadResourceScript(RT.java:363) at clojure.lang.RT.load(RT.java:453) at clojure.lang.RT.load(RT.java:419) at clojure.core$load$fn5677.invoke(core.clj:5893) at clojure.core$load.invokeStatic(core.clj:5892) at clojure.core$load.doInvoke(core.clj:5876) at clojure.lang.RestFn.invoke(RestFn.java:408) at clojure.core$load_one.invokeStatic(core.clj:5697) at clojure.core$load_one.invoke(core.clj:5692) at clojure.core$load_lib$fn5626.invoke(core.clj:5737) at clojure.core$load_lib.invokeStatic(core.clj:5736) at clojure.core$load_lib.doInvoke(core.clj:5717) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invokeStatic(core.clj:648) at clojure.core$load_libs.invokeStatic(core.clj:5778) at clojure.core$load_libs.doInvoke(core.clj:5758) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:648) at clojure.core$require.invokeStatic(core.clj:5796) at clojure.core$require.doInvoke(core.clj:5796) at clojure.lang.RestFn.invoke(RestFn.java:457) at matrix.core$eval20272$loading5569auto__20273.invoke(form-init3071421537802120931.clj:1) at matrix.core$eval20272.invokeStatic(form-init3071421537802120931.clj:1) at matrix.core$eval20272.invoke(form-init3071421537802120931.clj:1) at clojure.lang.Compiler.eval(Compiler.java:6927) at clojure.lang.Compiler.eval(Compiler.java:6916) at clojure.lang.Compiler.eval(Compiler.java:6890) at clojure.core$eval.invokeStatic(core.clj:3105) at clojure.core$eval.invoke(core.clj:3101) at clojure.main$repl$read_eval_print7408$fn7411.invoke(main.clj:240) at clojure.main$repl$read_eval_print7408.invoke(main.clj:240) at clojure.main$repl$fn7417.invoke(main.clj:258) at clojure.main$repl.invokeStatic(main.clj:258) at clojure.main$repl.doInvoke(main.clj:174) at clojure.lang.RestFn.invoke(RestFn.java:1523) at clojure.tools.nrepl.middleware.interruptible_eval$evaluate$fn941.invoke(interruptible_eval.clj:87) at clojure.lang.AFn.applyToHelper(AFn.java:152) at clojure.lang.AFn.applyTo(AFn.java:144) at clojure.core$apply.invokeStatic(core.clj:646) at clojure.core$with_bindingsSTAR.invokeStatic(core.clj:1881) at clojure.core$with_bindingsSTAR.doInvoke(core.clj:1881) at clojure.lang.RestFn.invoke(RestFn.java:425) at clojure.tools.nrepl.middleware.interruptible_eval$evaluate.invokeStatic(interruptible_eval.clj:85) at clojure.tools.nrepl.middleware.interruptible_eval$evaluate.invoke(interruptible_eval.clj:55) at clojure.tools.nrepl.middleware.interruptible_eval$interruptible_eval$fn986$fn989.invoke(interruptible_eval.clj:222) at clojure.tools.nrepl.middleware.interruptible_eval$run_next$fn__981.invoke(interruptible_eval.clj:190) at clojure.lang.AFn.run(AFn.java:22) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ---(end of nested stack traces)--- , compiling:(uncomplicate/neanderthal/opencl/clblast.clj:461:3)

amherag commented 8 years ago

@blueberry I was just thinking about that, and I found it:

    // Create a command-queue
    commandQueue = clCreateCommandQueue(
        context, devices[deviceIndex], 0, null);

It was written as devices[0], instead of devices[deviceIndex]. The same happens with the line String deviceName = getString(devices[deviceIndex], CL_DEVICE_NAME); which prints the info for the device.

Well, everything works now. I'll focus on tuning CLBlast for the K80. Thank you all!

gpu commented 8 years ago

Sorry about that. The snippet in the forum post originally only was a first, quick, basic test (actually I know that one should never underestimate how long a certain piece of code will stay alive). It should indeed have been device (or devices[deviceIndex]) instead of device[0]. I updated it in the forum post, and will also clean this up to become a sample for the website.

sundbp commented 8 years ago

This looks like what I'm running into on OSX. Just checked out master and tried running e.g. lein test, and get:

java.lang.UnsatisfiedLinkError: /private/var/folders/r6/1sgwwp6j2ys1bb3d_h6dwvc00000gn/T/libJOCLBlast_0_7_1-apple-x86_64.dylib: dlopen(/private/var/folders/r6/1sgwwp6j2ys1bb3d_h6dwvc00000gn/T/libJOCLBlast_0_7_1-apple-x86_64.dylib, 1): Library not loaded: libclblast.dylib
  Referenced from: /private/var/folders/r6/1sgwwp6j2ys1bb3d_h6dwvc00000gn/T/libJOCLBlast_0_7_1-apple-x86_64.dylib
  Reason: image not found

Reading this thread sounds like there's been a fix but no new release - any guidance as to when a new release is likely?

blueberry commented 8 years ago

Release will be shortly after the release of CLBlast and JOCLBlast. You can contact @CNugteren and @gpu for closer estimate, but it will probably be in a near future.

blueberry commented 8 years ago

@sundbp This issue will be fixed in the next release, which is due in a few days.

sundbp commented 8 years ago

Sweet, thanks.

On Tue, Jul 5, 2016 at 8:24 AM, Dragan Djuric notifications@github.com wrote:

@sundbp https://github.com/sundbp This issue will be fixed in the next release, which is due in a few days.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/uncomplicate/neanderthal/issues/15#issuecomment-230405189, or mute the thread https://github.com/notifications/unsubscribe/AADxJ5U5i-6ef-lYhdIwLVFcD4y-Vh1hks5qSgawgaJpZM4ImtEo .

gpu commented 8 years ago

The MacOS/JOCL specific part of this issue should be resolved in 0.8.0 (see https://github.com/gpu/JOCLBlast/issues/8#issuecomment-230835828 ) - hopefully it also works as expected in the more complex Clojure setup.

blueberry commented 8 years ago

@amherag @sundbp New release 0.7.0 is in Clojars. Please test whether it has resolved this issue (and if it hasn't, please reopen it).

sundbp commented 8 years ago

I pulled master on OSX and ran lein test. I see this failure: FAIL "3.1 Linear combination" at (ch03_test.clj:16) Expected: #RealBlockVector[double, n:5, stride:1](51.0 312.0 215.4 373.09999999999997 356.0) Actual: #RealBlockVector[double, n:5, stride:1](51.0 312.0 215.4 373.1 356.0) Exception in thread "main" java.lang.UnsupportedOperationException: The function clCreateCommandQueueWithProperties is not supported, compiling:(uncomplicate/neanderthal/examples/guides/tutorial_opencl_test.clj:117:1)

blueberry commented 8 years ago

Do not worry about that. This is obviously a floating point precision issue due a minor hardware difference between my AMD hardware and the one you have in your mac. What's important is that now the library is loaded properly.

Also, neanderthal does not use clojure test, but Midje, so it should be lein midje.

And I would need you or @amherag to actually test the library not by building from source (although it is nice too) but by using it as a dependency in a separate project.

Thanks for trying it out.

blueberry commented 8 years ago

@sundbp The UnsupportedOperationException is due to tests being written for OpenCL 2.0, while your Mac supports only 1.2. Please use the legacy namespace for creating command queues on Mac.

http://clojurecl.uncomplicate.org/codox/uncomplicate.clojurecl.legacy.html

sundbp commented 8 years ago

Ok, understand. Figured running the test suite would be the best way to "test" if it works :)

On Thu, Jul 7, 2016 at 10:07 AM, Dragan Djuric notifications@github.com wrote:

@sundbp https://github.com/sundbp The UnsupportedOperationException is due to tests being written for OpenCL 2.0, while your Mac supports only 1.2. Please use the legacy namespace for creating command queues on Mac.

http://clojurecl.uncomplicate.org/codox/uncomplicate.clojurecl.legacy.html

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/uncomplicate/neanderthal/issues/15#issuecomment-231024100, or mute the thread https://github.com/notifications/unsubscribe/AADxJ0QiPko2ZYMRmHj6MAkB2AwO8vTXks5qTMHZgaJpZM4ImtEo .