Closed eihli closed 3 years ago
You're probably using a mismatched version of javacpp-mkl. See the example hello world: https://github.com/uncomplicate/neanderthal/blob/master/examples/hello-world/project.clj
Ah yes. That was it. mkl-platform-redist
@ 2020.3-1.5.4
and neanderthal
@ 0.38.0
works. Thanks!
This doesn't work for me. I'm still getting the error. Here's my project.clj, the same one in hello-world example. I'm running the hello world example from the source:
(defproject hello-world "0.38.0"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies [[org.clojure/clojure "1.10.1"]
[uncomplicate/neanderthal "0.38.0"]
;;Optional. If bytedeco is not present, a system-wide MKL is used.
[org.bytedeco/mkl-platform-redist "2020.3-1.5.4"]]
;; Nvidia doesn't ship CUDA for macOS; you have to add this to your project
:exclusions [[org.jcuda/jcuda-natives :classifier "apple-x86_64"]
[org.jcuda/jcublas-natives :classifier "apple-x86_64"]]
;; If on Java 9+, you have to uncomment the following JVM option.
:jvm-opts ^:replace [#_"--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED"])
Full stack trace
More information.Execution error (UnsatisfiedLinkError) at java.lang.ClassLoader$NativeLibrary/load (ClassLoader.java:-2).
/private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/libneanderthal-mkl-0.33.07947871945054341194.jnilib: dlopen(/private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/libneanderthal-mkl-0.33.07947871945054341194.jnilib, 1): Library not loaded: @rpath/libmkl_intel_lp64.dylib
Referenced from: /private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/libneanderthal-mkl-0.33.07947871945054341194.jnilib
Reason: image not found
ERROR: Unhandled REPL handler exception processing message {:nrepl.middleware.print/stream? 1, :nrepl.middleware.print/print cider.nrepl.pprint/pprint, :nrepl.middleware.print/quota 1048576, :nrepl.middleware.print/options {:right-margin 80}, :op stacktrace, :session 0f1c7690-c341-4daa-97ce-c8c9d2e37dbc, :id 19}
java.lang.NoClassDefFoundError: Could not initialize class uncomplicate.neanderthal.internal.host.MKL
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at clojure.lang.RT.classForName(RT.java:2211)
at clojure.lang.RT.classForName(RT.java:2220)
at clojure.lang.Compiler.maybeResolveIn(Compiler.java:7438)
at clojure.core$ns_resolve.invokeStatic(core.clj:4370)
at clojure.core$ns_resolve.invokeStatic(core.clj:4359)
at clojure.core$ns_resolve.invoke(core.clj:4359)
at cider.nrepl.inlined_deps.orchard.v0v6v0.orchard.java$resolve_class$fn__42350.invoke(java.clj:310)
at cider.nrepl.inlined_deps.orchard.v0v6v0.orchard.java$resolve_class.invokeStatic(java.clj:310)
at cider.nrepl.inlined_deps.orchard.v0v6v0.orchard.java$resolve_class.invoke(java.clj:304)
at cider.nrepl.inlined_deps.orchard.v0v6v0.orchard.java$resolve_symbol.invokeStatic(java.clj:344)
at cider.nrepl.inlined_deps.orchard.v0v6v0.orchard.java$resolve_symbol.invoke(java.clj:330)
at cider.nrepl.middleware.stacktrace$analyze_fn.invokeStatic(stacktrace.clj:84)
at cider.nrepl.middleware.stacktrace$analyze_fn.invoke(stacktrace.clj:63)
at clojure.core$comp$fn__5807.invoke(core.clj:2569)
at clojure.core$comp$fn__5807.invoke(core.clj:2569)
at clojure.core$comp$fn__5807.invoke(core.clj:2569)
at cider.nrepl.middleware.stacktrace$analyze_frame.invokeStatic(stacktrace.clj:180)
at cider.nrepl.middleware.stacktrace$analyze_frame.invoke(stacktrace.clj:177)
at clojure.core$map$fn__5866.invoke(core.clj:2755)
at clojure.lang.LazySeq.sval(LazySeq.java:42)
at clojure.lang.LazySeq.seq(LazySeq.java:51)
at clojure.lang.RT.seq(RT.java:535)
at clojure.core$seq__5402.invokeStatic(core.clj:137)
at clojure.core$map$fn__5870.invoke(core.clj:2757)
at clojure.lang.LazySeq.sval(LazySeq.java:42)
at clojure.lang.LazySeq.seq(LazySeq.java:51)
at clojure.lang.Cons.next(Cons.java:39)
at clojure.lang.RT.next(RT.java:713)
at clojure.core$next__5386.invokeStatic(core.clj:64)
at clojure.core.protocols$fn__8159.invokeStatic(protocols.clj:169)
at clojure.core.protocols$fn__8159.invoke(protocols.clj:124)
at clojure.core.protocols$fn__8114$G__8109__8123.invoke(protocols.clj:19)
at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
at clojure.core.protocols$fn__8146.invokeStatic(protocols.clj:75)
at clojure.core.protocols$fn__8146.invoke(protocols.clj:75)
at clojure.core.protocols$fn__8088$G__8083__8101.invoke(protocols.clj:13)
at clojure.core$reduce.invokeStatic(core.clj:6828)
at clojure.core$into.invokeStatic(core.clj:6895)
at clojure.core$mapv.invokeStatic(core.clj:6903)
at clojure.core$mapv.invoke(core.clj:6903)
at cider.nrepl.middleware.stacktrace$flag_duplicates.invokeStatic(stacktrace.clj:175)
at cider.nrepl.middleware.stacktrace$flag_duplicates.invoke(stacktrace.clj:164)
at cider.nrepl.middleware.stacktrace$analyze_stacktrace.invokeStatic(stacktrace.clj:186)
at cider.nrepl.middleware.stacktrace$analyze_stacktrace.invoke(stacktrace.clj:182)
at cider.nrepl.middleware.stacktrace$analyze_cause.invokeStatic(stacktrace.clj:290)
at cider.nrepl.middleware.stacktrace$analyze_cause.invoke(stacktrace.clj:281)
at cider.nrepl.middleware.stacktrace$analyze_causes$fn__43515.invoke(stacktrace.clj:315)
at clojure.core$map$fn__5862$fn__5863.invoke(core.clj:2742)
at clojure.core$take_while$fn__5913$fn__5914.invoke(core.clj:2901)
at clojure.lang.Iterate.reduce(Iterate.java:81)
at clojure.core$transduce.invokeStatic(core.clj:6883)
at clojure.core$into.invokeStatic(core.clj:6899)
at clojure.core$into.invoke(core.clj:6887)
at cider.nrepl.middleware.stacktrace$analyze_causes.invokeStatic(stacktrace.clj:314)
at cider.nrepl.middleware.stacktrace$analyze_causes.invoke(stacktrace.clj:306)
at cider.nrepl.middleware.stacktrace$handle_stacktrace.invokeStatic(stacktrace.clj:323)
at cider.nrepl.middleware.stacktrace$handle_stacktrace.invoke(stacktrace.clj:320)
at clojure.lang.Var.invoke(Var.java:388)
at cider.nrepl$wrap_stacktrace$fn__35396.invoke(nrepl.clj:410)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_debug$fn__35298.invoke(nrepl.clj:152)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_enlighten$fn__35306.invoke(nrepl.clj:178)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_format$fn__35314.invoke(nrepl.clj:182)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_info$fn__35322.invoke(nrepl.clj:196)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_out$fn__35356.invoke(nrepl.clj:312)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_resource$fn__35380.invoke(nrepl.clj:384)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_clojuredocs$fn__35454.invoke(nrepl.clj:504)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_undef$fn__35428.invoke(nrepl.clj:471)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at nrepl.middleware.load_file$wrap_load_file$fn__34642.invoke(load_file.clj:81)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_content_type$fn__35256.invoke(nrepl.clj:98)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_ns$fn__35348.invoke(nrepl.clj:278)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_inspect$fn__35332.invoke(nrepl.clj:215)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at nrepl.middleware.caught$wrap_caught$fn__34550.invoke(caught.clj:97)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_apropos$fn__35272.invoke(nrepl.clj:118)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_test$fn__35404.invoke(nrepl.clj:420)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at nrepl.middleware.print$wrap_print$fn__34517.invoke(print.clj:234)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_classpath$fn__35280.invoke(nrepl.clj:126)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_trace$fn__35412.invoke(nrepl.clj:442)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_tracker$fn__35420.invoke(nrepl.clj:460)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at nrepl.middleware.session$session$fn__34738.invoke(session.clj:272)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_profile$fn__35364.invoke(nrepl.clj:321)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at cider.nrepl$wrap_slurp$fn__35264.invoke(nrepl.clj:110)
at nrepl.middleware$wrap_conj_descriptor$fn__34293.invoke(middleware.clj:16)
at nrepl.server$handle_STAR_.invokeStatic(server.clj:18)
at nrepl.server$handle_STAR_.invoke(server.clj:15)
at nrepl.server$handle$fn__34775.invoke(server.clj:27)
at clojure.core$binding_conveyor_fn$fn__5754.invoke(core.clj:2030)
at clojure.lang.AFn.call(AFn.java:18)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Syntax error compiling at (uncomplicate/neanderthal/native.clj:1:1).
No namespace: uncomplicate.neanderthal.internal.host.mkl
Alright I've got a lead on this but I'm in way over my head.
So uncomplicate.neanderthal.native
requires uncomplicate.neanderthal.internal.host.mkl
. The problem is running (require '[uncomplicate.neanderthal.internal.host.mkl])
fails with something like:
Execution error (UnsatisfiedLinkError) at java.lang.ClassLoader$NativeLibrary/load0 (ClassLoader.java:-2).
/private/var/folders/8m/p0_shyw51mq8mf2bb7lg52yc0000gn/T/libneanderthal-mkl-0.33.010233505079158403947.jnilib: dlopen(/private/var/folders/8m/p0_shyw51mq8mf2b
b7lg52yc0000gn/T/libneanderthal-mkl-0.33.010233505079158403947.jnilib, 1): Library not loaded: @rpath/libmkl_intel_lp64.dylib
Referenced from: /private/var/folders/8m/p0_shyw51mq8mf2bb7lg52yc0000gn/T/libneanderthal-mkl-0.33.010233505079158403947.jnilib
Reason: image not found
After many searches and being generally out of my league I end up reading this https://blog.krzyzanowskim.com/2018/12/05/rpath-what/.
In that article I find that I can use otool -l
to find out what this @rpath
thing actually resolves to, it's tricky though because the binary /private/var/folders/8m/p0_shyw51mq8mf2bb7lg52yc0000gn/T/libneanderthal-mkl-0.33.010233505079158403947.jnilib
only exists while my REPL is running and after I've tried the failing require. So in another shell I run otool -l /private/var/folders/8m/p0_shyw51mq8mf2bb7lg52yc0000gn/T/libneanderthal-mkl-0.33.010233505079158403947.jnilib | grep -B 1 -A 2 LC_RPATH
and I get this:
Load command 15
cmd LC_RPATH
cmdsize 72
path /opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib (offset 12)
Which makes me think I wouldn't have issues if I installed MKL 2018.3.185 on my system, but I didn't, I'm using org.bytedeco/mkl-platform-redist {:mvn/version "2020.3-1.5.4"}
. Seems strange that this rpath was set to such a specific install location, I'm not sure when this linking happens in Clojure/Java land or who has control over it.
Via previous searches and debug logs etc. I had already discovered that the dylib's are actually in ~/.javacpp/cache/mkl-2020.3-1.5.4-macosx-x86_64-redist.jar/org/bytedeco/mkl/macosx-x86_64/
so I simply tried copying them with:
sudo mkdir -p /opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib
sudo cp ~/.javacpp/cache/mkl-2020.3-1.5.4-macosx-x86_64-redist.jar/org/bytedeco/mkl/macosx-x86_64/ /opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib
And low and behold I can now (require '[uncomplicate.neanderthal.internal.host.mkl])
as well as the original goal which was (require '[uncomplicate.neanderthal.nativel])
. Running (uncomplicate.neanderthal.native/dge 3 2 [1 2 3 4 5 6])
works as well.
I don't know where to go next as to tracking down what a more proper solution is than my copying of the dylib's.
cc people who seemed to have cared about this issue (or a potentially related one) in the past since conversations are spread out in multiple places: @blueberry @zendevil @jasonwaack @lccambiaghi @Hendekagon @SolbiatiAlessandro
I don't know why you did all this, nor whether you've read the Getting Started guide, but it's great that you solved your issue at the end.
For posterity: I recommend reading Getting Started and proceeding accordingly.
Based on this issue and #31 I gathered that people were having trouble getting it to work in anyway besides installing mkl on the system and disabling SIP and exporting the right DYLD_LIBRARY_PATH. Yes I did read the starting guide, it claims one should be able to just include the bytedeco dependency:
Final note If you prefer zero-install, just include [org.bytedeco/mkl-platform-redist "2020.3-1.5.4"] as a dependencly in your leiningen project and none of these is necessary.
But this statement is not true (at least as I interpreted it), you still need to copy the dylib's to the right place, either the one I used or perhaps /usr/local/lib which I haven't tried yet. The starting guide mentions making a copy to /usr/local/lib but I was under the impression based on the quote above that such would only be necessary for a system install of mkl and not when including the bytedeco dependency.
Based on this issue and #31 I gathered that people were having trouble getting it to work in anyway besides installing mkl on the system and disabling SIP and exporting the right DYLD_LIBRARY_PATH. Yes I did read the starting guide, it claims one should be able to just include the bytedeco dependency:
Final note If you prefer zero-install, just include [org.bytedeco/mkl-platform-redist "2020.3-1.5.4"] as a dependencly in your leiningen project and none of these is necessary.
But this statement is not true (at least as I interpreted it), you still need to copy the dylib's to the right place, either the one I used or perhaps /usr/local/lib which I haven't tried yet. The starting guide mentions making a copy to /usr/local/lib but I was under the impression based on the quote above that such would only be necessary for a system install of mkl and not when including the bytedeco dependency.
You have to disable SIP, then everything works in the same way as linux, and that's it. Now, I understand that some people might have reasons why they don't want to disable SIP, but that's beyond what I can support. It's literally that there is a big Apple dam blocking the river, and people are complaining why there is no water downstream. There is no water because the dam (SIP) is blocking all water. If you don't want to open the dam, that's ok, but then I don't know how to help.
It's very possible that I'm still just missing something and I apologize if I'm being confusing in how I try to explain this.
I tried disabling SIP and including the bytedeco dependency but I still had the Reason: image not found
error. Would you have expected those two steps alone to be sufficient?
It should be. Please try the Hello World example project first https://github.com/uncomplicate/neanderthal/tree/master/examples/hello-world Please read the comments in project.clj regarding Java 9+ and jvm-opts.
If that doesn't work, then it may be that there's something set up on your OS that messes with this, or Apple introduced yet another roadblock for developers. IDK. I only have an old MacBook Air, and no access to latest macOS. But I've helped quite a few people set up new macs and in each case when they had various issues the source of the problem was always SIP. Maybe that reconfiguration requires a reboot, who knows...
I've tried hello-world, using OpenJDK 11.0.9.1 and thus uncommenting the jvm-opts, I disabled SIP and rebooted, it gives the same error
Execution error (UnsatisfiedLinkError) at java.lang.ClassLoader$NativeLibrary/load0 (ClassLoader.java:-2).
/private/var/folders/8m/p0_shyw51mq8mf2bb7lg52yc0000gn/T/libneanderthal-mkl-0.33.013592419731879182342.jnilib: dlopen(/private/var/folders/8m/p0_shyw51mq8mf2bb7lg52yc0000gn/T/libneanderthal-mkl-0.33.013592419731879182342.jnilib, 1): Library not loaded: @rpath/libmkl_intel_lp64.dylib
Referenced from: /private/var/folders/8m/p0_shyw51mq8mf2bb7lg52yc0000gn/T/libneanderthal-mkl-0.33.013592419731879182342.jnilib
Reason: image not found
I appreciate your responses and don't expect you to resolve this for me. If you don't mind, I'm wondering if you know any details about this file and where it comes from /private/var/folders/8m/p0_shyw51mq8mf2bb7lg52yc0000gn/T/libneanderthal-mkl-0.33.013592419731879182342.jnilib
. It seems to be created once I attempt to require uncomplicate.neanderthal.internal.host.mkl and it is the library that when loaded by dlopen fails to find it's dependency on the mkl dylib's.
Then it seems that this deserves its own issue. Please create a new with as many detailed info as you can share. I really don't know what could be the problem, nor I have a machine where I could reproduce this, but someone else might help. In the meantime, I guess some other installation option might be a temporary workaround.
@burbma I don't know if you have solved this, but I ran into the same problem and was able to fix it. It seems like the mkl redist library is missing libmkl_intel_lp64.dylib file which is supposed to link to libmkl_intel_lp64.1.dylib. I just manually created the link in the same folder in addition to creating a link of /opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib that points to the ~/.javacpp/cache/mkl-2020.3-1.5.4-macosx-x86_64-redist.jar/org/bytedeco/mkl/macosx-x86_64/. That seems to fixed the problem.
@blueberry I think the redist mkl library is supposed to work out of the box, but I think they missed a hard link which requires user to manually set it up.
I also could not get the redist mkl library to work however installing the intel libs worked the first try, I did the following (macos):
install intel mkl via gui (make a note of version) link
set DYLD_LIBRARY_PATH
export DYLD_LIBRARY_PATH=/opt/intel/oneapi/mkl/VERSION_FROM_STEP_1/lib
try lein repl
➛ lein repl
OpenJDK 64-Bit Server VM warning: -XX:+UseLargePages not supported in this VM
nREPL server started on port 55698 on host 127.0.0.1 - nrepl://127.0.0.1:55698
REPL-y 0.5.1, nREPL 0.8.3
Clojure 1.10.3
OpenJDK 64-Bit Server VM 17+0
Docs: (doc function-name-here)
(find-doc "part-of-name-here")
Source: (source function-name-here)
Javadoc: (javadoc java-object-or-class-here)
Exit: Control+D or (exit) or (quit)
Results: Stored in vars *1, *2, *3, an exception in *e
user=> (ns hello-world.native
#_=> (:require [uncomplicate.neanderthal
#_=> [core :refer :all]
#_=> [native :refer :all]]))
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
nil
hello-world.native=> (def a (dge 2 3 [1 2 3 4 5 6]))
#'hello-world.native/a
hello-world.native=> (def b (dge 3 2 [1 3 5 7 9 11]))
#'hello-world.native/b
hello-world.native=> (mm a b)
#RealGEMatrix[double, mxn:2x2, layout:column, offset:0]
▥ ↓ ↓ ┓
→ 35.00 89.00
→ 44.00 116.00
┗ ┛
2021.1 mkl no longer available on intel site, can download here and set DYLD_LIBRARY_PATH
as above
After reading the docs at https://neanderthal.uncomplicate.org/articles/getting_started.html#the-native-library-used-by-neanderthals-native-engine-optional I expected I wouldn't see an error about missing mkl-related files if I included the bytedeco dep.
But I'm getting this when I launch cider.
deps.edn