Library `neanderthal-mkl-0.23.0.dll` not found!

I have been using neanderthal for about a year for my personal project. I had been using clojure 1.8.0 and neanderthal 0.22.0 on my Windows 10 (and also Fedora Linux, but this issue shall focus on Windows 10).

I had a problem with my old laptop and had to re-setup my clojure environment on my current laptop. Everything clojure related worked except neanderthal. First there was the issue with jcuda and jcublas (my work does not use GPU computing) which was annoying but it got fixed looking at the hello-world example; not to mention most advices here are Mac based so I had to tinker around for Windows.

My project.clj looks like this

(defproject test "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "EPL-2.0 OR GPL-2.0-or-later WITH Classpath-exception-2.0"
            :url "https://www.eclipse.org/legal/epl-2.0/"}
  :dependencies [[org.clojure/clojure "1.10.0"]
                           [uncomplicate/neanderthal "0.23.1"]]
  :exclusions [org.jcuda/jcuda-natives org.jcuda/jcublas-natives]
  :plugins [[lein-with-env-vars "0.2.0"]]
  :env-vars {:PATH "C:\\Program Files\\clojure\\dll_for_neanderthal"}
  :repl-options {:init-ns test.core})

Anyways, I can now start neanderthal as shown below

>lein with-env-vars repl

Then,

test.core=> (ns example
       #_=>   (:use [uncomplicate.neanderthal core native]))
nil
example=> (def x (dv 1 2 3))
#'example/x
example=> (def y (dv 10 20 30))

It's just that currently it's practically unusable because when I call

#'example/y
example=> (dot x y)

it returns

java.lang.RuntimeException: Library 'neanderthal-mkl-0.23.0.dll' not found!
        at uncomplicate.neanderthal.internal.host.NarSystem.getLibPath(NarSystem.java:172)
        at uncomplicate.neanderthal.internal.host.NarSystem.loadLibrary(NarSystem.java:45)
        at uncomplicate.neanderthal.internal.host.CBLAS.<clinit>(CBLAS.java:29)
        at uncomplicate.neanderthal.internal.host.mkl.DoubleVectorEngine.dot(mkl.clj:423)
        at uncomplicate.neanderthal.core$dot.invokeStatic(core.clj:886)
        at uncomplicate.neanderthal.core$dot.invoke(core.clj:875)
        at example$eval29001.invokeStatic(form-init6432747691677904219.clj:1)
        at example$eval29001.invoke(form-init6432747691677904219.clj:1)
        at clojure.lang.Compiler.eval(Compiler.java:7176)
        at clojure.lang.Compiler.eval(Compiler.java:7131)
        at clojure.core$eval.invokeStatic(core.clj:3214)
        at clojure.core$eval.invoke(core.clj:3210)
        at clojure.main$repl$read_eval_print__9068$fn__9071.invoke(main.clj:414)
        at clojure.main$repl$read_eval_print__9068.invoke(main.clj:414)
        at clojure.main$repl$fn__9077.invoke(main.clj:435)
        at clojure.main$repl.invokeStatic(main.clj:435)
        at clojure.main$repl.doInvoke(main.clj:345)
        at clojure.lang.RestFn.invoke(RestFn.java:1523)
        at nrepl.middleware.interruptible_eval$evaluate.invokeStatic(interruptible_eval.clj:79)
        at nrepl.middleware.interruptible_eval$evaluate.invoke(interruptible_eval.clj:55)
        at nrepl.middleware.interruptible_eval$interruptible_eval$fn__939$fn__943.invoke(interruptible_eval.clj:142)
        at clojure.lang.AFn.run(AFn.java:22)
        at nrepl.middleware.session$session_exec$main_loop__1040$fn__1044.invoke(session.clj:171)
        at nrepl.middleware.session$session_exec$main_loop__1040.invoke(session.clj:170)
        at clojure.lang.AFn.run(AFn.java:22)
        at java.lang.Thread.run(Unknown Source)
Execution error at uncomplicate.neanderthal.internal.host.NarSystem/getLibPath (NarSystem.java:172).
Library 'neanderthal-mkl-0.23.0.dll' not found!

So I went back and checked my Intel MKL installation, checked my path and made sure I got the listed libraries. I also made sure I ran mklvars.bat. However none of these efforts fixed it.

I then wondered if the reason I could not run neanderthal was because I had installed 2019 MKL because

Neanderthal has been built and tested with Intel MKL 2018; please make sure that you use a compatible MKL version.

So I uninstalled the 2019 MKL and installed the MKL 2018.4.274. This still did not fix it.

Why am I getting ?

java.lang.RuntimeException: Library 'neanderthal-mkl-0.23.0.dll' not found!

i.e., why is it searching for neanderthal-mkl-0.23.0.dll? and how do I fix this? (so than I can continue using neanderthal like before)

If you switch the version back to 0.22.1, does it work or not?
Can you show the contents of /.m2/repository/uncomplicate/neanderthal-native/neanderthal-native-0.23.1.jar and in particular the contents of /lib/amd64-Windows-msvc/jni/ inside that jar?
You may have a corrupted download. Remove uncomplicate folder from .m2 and let leiningen re-download it.

Thanks for reply.

If you switch the version back to 0.22.1, does it work or not?

No. It's not working. I get a similar error but for neanderthal-mkl-0.21.0.dll

java.lang.RuntimeException: Library 'neanderthal-mkl-0.21.0.dll' not found!
        at uncomplicate.neanderthal.internal.host.NarSystem.getLibPath(NarSystem.java:172)
        at uncomplicate.neanderthal.internal.host.NarSystem.loadLibrary(NarSystem.java:45)
        at uncomplicate.neanderthal.internal.host.CBLAS.<clinit>(CBLAS.java:29)
        at uncomplicate.neanderthal.internal.host.mkl.DoubleVectorEngine.dot(mkl.clj:385)
        at uncomplicate.neanderthal.core$dot.invokeStatic(core.clj:886)
        at uncomplicate.neanderthal.core$dot.invoke(core.clj:875)
        at example$eval28756.invokeStatic(form-init8121861227976999100.clj:1)
        at example$eval28756.invoke(form-init8121861227976999100.clj:1)
        at clojure.lang.Compiler.eval(Compiler.java:7176)
        at clojure.lang.Compiler.eval(Compiler.java:7131)
        at clojure.core$eval.invokeStatic(core.clj:3214)
        at clojure.core$eval.invoke(core.clj:3210)
        at clojure.main$repl$read_eval_print__9068$fn__9071.invoke(main.clj:414)
        at clojure.main$repl$read_eval_print__9068.invoke(main.clj:414)
        at clojure.main$repl$fn__9077.invoke(main.clj:435)
        at clojure.main$repl.invokeStatic(main.clj:435)
        at clojure.main$repl.doInvoke(main.clj:345)
        at clojure.lang.RestFn.invoke(RestFn.java:1523)
        at nrepl.middleware.interruptible_eval$evaluate.invokeStatic(interruptible_eval.clj:79)
        at nrepl.middleware.interruptible_eval$evaluate.invoke(interruptible_eval.clj:55)
        at nrepl.middleware.interruptible_eval$interruptible_eval$fn__939$fn__943.invoke(interruptible_eval.clj:142)
        at clojure.lang.AFn.run(AFn.java:22)
        at nrepl.middleware.session$session_exec$main_loop__1040$fn__1044.invoke(session.clj:171)
        at nrepl.middleware.session$session_exec$main_loop__1040.invoke(session.clj:170)
        at clojure.lang.AFn.run(AFn.java:22)
        at java.lang.Thread.run(Unknown Source)
Execution error at uncomplicate.neanderthal.internal.host.NarSystem/getLibPath (NarSystem.java:172).
Library 'neanderthal-mkl-0.21.0.dll' not found!

Can you show the contents of /.m2/repository/uncomplicate/neanderthal-native/neanderthal-native-0.23.1.jar and in particular the contents of /lib/amd64-Windows-msvc/jni/ inside that jar?

My .m2\repository\uncomplicate\neanderthal-native\0.23.1 contains

_remote.repositories
neanderthal-native-0.23.1.jar
neanderthal-native-0.23.1.jar.sha1
neanderthal-native-0.23.1.pom
neanderthal-native-0.23.1.pom.sha1

Within neanderthal-native-0.23.1.jar and under /lib/amd64-Windows-msvc/jni/ are the five files

history.xml
neanderthal-mkl-0.23.0.dll
neanderthal-mkl-0.23.0.dll.manifest
neanderthal-mkl-0.23.0.exp
neanderthal-mkl-0.23.0.lib

So I actually have neanderthal-mkl-0.23.0.dll the file!

You may have a corrupted download. Remove uncomplicate folder from .m2 and let leiningen re-download it.

I deleted ~ /.m2/repository/uncomplicate/ but it still does not work.

Then it seems to me that you messed up the MKL setup on your machine, since even the old versionof neanderthal is not working.

After spending time on this for the past few days and the whole night I am giving up for now.

What I have tried:

Installed various versions of Intel MKL: 2019.4.245, 2019.3.203, 2019.0.117, 2018.4.274, 2018.2.185 (installed in the past working setup) and 2018.0.124
Passed all possible environment paths under C:\Program Files (x86)\IntelSWTools into > Advanced system settings > EnvironmentVariables > System variables > New
Delete ~/.m2/uncomplicate - reinstall neanderthal (versions 0.22.1 (previously worked) and 0.23.1 (current))
even used old lein.bat from my previously working setup: clojure 1.8, neanderthal 0.22.1, MKL 2018.2.185.

From all these trial and error and I think MKL setup is unlikely to be the cause. Among all the clojure packages that I use neanderthal is the only one giving me this error.

I don't know much about how clojars, maven and java work but I can only speculate on some potential reasons:

the uncomplicate repository in clojars or the way maven handles them has changed or broke (because I am unable to run neanderthal with the past working neanderthal setup, i.e., 0.22.1)
or its java and the way it is looking for paths is resulting in this error. I have Java 8 update 211. (I ran with and without :jvm-opts ^:replace [#_"--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED"])

Anyway I will be unable to use neanderthal for a while, which is a shame because I had couple of my projects depending on it.

Oh well! I will try to get back on this later (I just can't think of anything else now).

Installed various versions of Intel MKL: 2019.4.245, 2019.3.203, 2019.0.117, 2018.4.274, 2018.2.185 (installed in the past working setup) and 2018.0.124

This should not make any difference. Any of these is compatible with neanderthal.

Passed all possible environment paths under C:\Program Files (x86)\IntelSWTools into > Advanced system settings > EnvironmentVariables > System variables > New

From all these trial and error and I think MKL setup is unlikely to be the cause. Among all the clojure packages that I use neanderthal is the only one giving me this error.

Does any of these Clojure libraries use MKL?

I don't know much about how clojars, maven and java work but I can only speculate on some potential reasons:

the uncomplicate repository in clojars or the way maven handles them has changed or broke (because I am unable to run neanderthal with the past working neanderthal setup, i.e., 0.22.1)

This is extremely unlikely (or even impossible) since the libraries in the Maven/Clojars repositories are immutable, and the way maven handles them is rather stable for the last 10+ years.

Oh well! I will try to get back on this later (I just can't think of anything else now).

If you get back to this, feel free to reopen the issue.

uncomplicate / neanderthal

Library `neanderthal-mkl-0.23.0.dll` not found! #61