Problems to load Python 3.8 under Windows 10 (possible dynamic variable binding problem)

[Goldritter]

Hi,

I tried to initialize libpython-clj (Version "1.46") for my Python 3.8.6 version which is installed under C:\opt\Python\Python3.8 with following code: (py/initialize! :python-executable "C:/opt/Python/Python3.8" :library-path "C:/opt/Python/Python3.8/libs/python38.lib")

and get an error that the library cannot be loaded.
As you can see in the logging info and the Stack trace at the end the correct paths and Python version are correctly logged, but in the Stacktrace the used Python version is suddenly 3.6.

I've looked further into libpython-clj.jna.base and tech.jna.base and it looks like the dynamic variables python-library and python-library-names defined in libpython-clj.jna.base have no correct binding when the function tech.jna.base/load-library is called. If I execute (tech.jna.base/load-library "python38") the library is correctly loaded without any Exception (tech.jna.base/load-library "python38") => #object[com.sun.jna.NativeLibrary 0x19b2ebd8 "Native Library <python38.dll@140728377081856>"] .

Is this really a problem with the correct binding or do I miss something and/or forgot to add specific parameters during the initialization?

Best Goldritter

Loggin info:

14.09.2020 10:01:23,051 INFO logging:326 - Executing python initialize with options:{:python-executable "C:/opt/Python/Python3.8", :program-name nil, :python-home nil, :library-path "C:/opt/Python/Python3.8/libs/python38.lib"} 14.09.2020 10:01:23,052 INFO logging:326 - Detecting startup-info for Python executable: C:/opt/Python/Python3.8 14.09.2020 10:01:23,808 INFO interpreter:326 - Startup info detected: {:lib-version "3.8", :java-library-path-addendum "C:\opt\Python\Python3.8\lib", :exec-prefix "C:\opt\Python\Python3.8", :executable "C:\opt\Python\Python3.8\python.exe", :libnames ("C:/opt/Python/Python3.8/libs/python38.lib" "python3.8"), :prefix "C:\opt\Python\Python3.8", :base-prefix "C:\opt\Python\Python3.8", :base-exec-prefix "C:\opt\Python\Python3.8", :python-home "C:\opt\Python\Python3.8", :version [3 8 6], :platform "win32"}

14.09.2020 10:01:23,811 INFO interpreter:326 - Trying python library names ["C:/opt/Python/Python3.8/libs/python38.lib" "C:/opt/Python/Python3.8/libs/python38.lib" "python3.8" "python3.7m" "python3.6m"] 14.09.2020 10:01:23,812 INFO interpreter:326 - Setting java library path: C:\opt\Python\Python3.8\lib:C:\opt\Python\Python3.8\lib:C:\opt\Python\Python3.8\lib:C:\opt\Python\Python3.8\lib:C:\opt\Python\Python3.8\lib:C:\opt\Python\Python3.8\lib:C:\opt\Python\Python3.8\lib:C:\opt\Python\Python3.8\lib: ;.

Stacktrace:

Syntax error compiling at (<...>/src/python/config.clj:4:1). Failed to load library at clojure.lang.Compiler.load(Compiler.java:7647) clojure.lang.ExceptionInfo: Failed to load library {:libname "python3.6m", :paths [[:system ["python3.6m"]] [:java-library-path ["C\python3.6m.dll" "\opt\Python\Python3.8\lib\python3.6m.dll" "C\python3.6m.dll" "\opt\Python\Python3.8\lib\python3.6m.dll" "C\python3.6m.dll" "\opt\Python\Python3.8\lib\python3.6m.dll" "C\python3.6m.dll" "\opt\Python\Python3.8\lib\python3.6m.dll" "C\python3.6m.dll" "\opt\Python\Python3.8\lib\python3.6m.dll" "C\python3.6m.dll" "\opt\Python\Python3.8\lib\python3.6m.dll" "C\python3.6m.dll" "\opt\Python\Python3.8\lib\python3.6m.dll" "C\python3.6m.dll" < ommited many other path combinations with */ python3.6m.dll > ]]]} at tech.jna.base$do_load_library.invokeStatic(base.clj:158) at tech.jna.base$do_load_library.invoke(base.clj:123) at tech.jna.base$load_library.invokeStatic(base.clj:178) at tech.jna.base$load_library.invoke(base.clj:174) at tech.jna$load_library.invokeStatic(jna.clj:87) at tech.jna$load_library.invoke(jna.clj:85) at libpython_clj.python.interpreter$setup_direct_mappingBANG.invokeStatic(interpreter.clj:430) at libpython_clj.python.interpreter$setup_direct_mappingBANG.invoke(interpreter.clj:428) at libpython_clj.python.interpreter$initializeBANG.invokeStatic(interpreter.clj:464) at libpython_clj.python.interpreter$initializeBANG.doInvoke(interpreter.clj:434) at clojure.lang.RestFn.invoke(RestFn.java:619) at libpython_clj.python$initializeBANG.invokeStatic(python.clj:242) at libpython_clj.python$initializeBANG.doInvoke(python.clj:229) at clojure.lang.RestFn.invoke(RestFn.java:457) at python.config$eval35333.invokeStatic(config.clj:5) at python.config$eval35333.invoke(config.clj:5) at clojure.lang.Compiler.eval(Compiler.java:7176) at clojure.lang.Compiler.load(Compiler.java:7635) at python.config$eval35321.invokeStatic(form-init16284440531770716747.clj:1) at python.config$eval35321.invoke(form-init16284440531770716747.clj:1) at clojure.lang.Compiler.eval(Compiler.java:7176) at clojure.lang.Compiler.eval(Compiler.java:7131) at clojure.core$eval.invokeStatic(core.clj:3214) at clojure.core$eval.invoke(core.clj:3210) at clojure.main$repl$read_eval_print9068$fn9071.invoke(main.clj:414) at clojure.main$repl$read_eval_print9068.invoke(main.clj:414) at clojure.main$repl$fn9077.invoke(main.clj:435) at clojure.main$repl.invokeStatic(main.clj:435) at clojure.main$repl.doInvoke(main.clj:345) at clojure.lang.RestFn.invoke(RestFn.java:1523) at nrepl.middleware.interruptible_eval$evaluate.invokeStatic(interruptible_eval.clj:79) at nrepl.middleware.interruptible_eval$evaluate.invoke(interruptible_eval.clj:55) at nrepl.middleware.interruptible_eval$interruptible_eval$fn34601$fn34605.invoke(interruptible_eval.clj:142) at clojure.lang.AFn.run(AFn.java:22) at nrepl.middleware.session$session_exec$main_loop34702$fn34706.invoke(session.clj:171) at nrepl.middleware.session$session_exec$main_loop__34702.invoke(session.clj:170) at clojure.lang.AFn.run(AFn.java:22) at java.base/java.lang.Thread.run(Thread.java:835)

[jjtolton]

Heya Marcus! Not sure if I’m reading this correctly, but the stack trace says platform="win32". Is this a 32 but system?

On Mon, Sep 14, 2020 at 4:41 AM Marcus Lindner notifications@github.com wrote:

Hi,

I tried to initialize libpython-clj (Version "1.46") for my Python 3.8.6 version which is installed under C:\opt\Python\Python3.8 with following code:

(py/initialize! :python-executable "C:/opt/Python/Python3.8" :library-path "C:/opt/Python/Python3.8/libs/python38.lib")

and get an error that the library cannot be loaded.

As you can see in the logging info and the Stack trace at the end the correct paths and Python version are correctly logged, but in the Stacktrace the used Python version is suddenly 3.6.

I've looked further into libpython-clj.jna.base and tech.jna.base and it looks like the dynamic variables python-library and python-library-names defined in libpython-clj.jna.base have no correct binding when the function tech.jna.base/load-library is called. If I execute (tech.jna.base/load-library "python38") the library is correctly loaded without any Exception

(tech.jna.base/load-library "python38") =>

object[com.sun.jna.NativeLibrary 0x19b2ebd8 "Native Library

python38.dll@140728377081856"]

.

Is this really a problem with the correct binding or do I miss something and/or forgot to add specific parameters during the initialization?

Best

Marcus

Loggin info:

14.09.2020 10:01:23,051 INFO logging:326 - Executing python initialize with options:{:python-executable "C:/opt/Python/Python3.8", :program-name nil, :python-home nil, :library-path "C:/opt/Python/Python3.8/libs/python38.lib"}

14.09.2020 10:01:23,052 INFO logging:326 - Detecting startup-info for Python executable: C:/opt/Python/Python3.8

14.09.2020 10:01:23,808 INFO interpreter:326 - Startup info detected:

{:lib-version "3.8",

:java-library-path-addendum "C:\opt\Python\Python3.8\lib",

:exec-prefix "C:\opt\Python\Python3.8",

:executable "C:\opt\Python\Python3.8\python.exe",

:libnames ("C:/opt/Python/Python3.8/libs/python38.lib" "python3.8"),

:prefix "C:\opt\Python\Python3.8",

:base-prefix "C:\opt\Python\Python3.8",

:base-exec-prefix "C:\opt\Python\Python3.8",

:python-home "C:\opt\Python\Python3.8",

:version [3 8 6],

:platform "win32"}

14.09.2020 10:01:23,811 INFO interpreter:326 - Trying python library names ["C:/opt/Python/Python3.8/libs/python38.lib" "C:/opt/Python/Python3.8/libs/python38.lib" "python3.8" "python3.7m" "python3.6m"]

14.09.2020 10:01:23,812 INFO interpreter:326 - Setting java library path: C:\opt\Python\Python3.8\lib:C:\opt\Python\Python3.8\lib:C:\opt\Python\Python3.8\lib:C:\opt\Python\Python3.8\lib:C:\opt\Python\Python3.8\lib:C:\opt\Python\Python3.8\lib:C:\opt\Python\Python3.8\lib:C:\opt\Python\Python3.8\lib: ;.

Stacktrace:

Syntax error compiling at (<...>/src/python/config.clj:4:1).

Failed to load library

at clojure.lang.Compiler.load(Compiler.java:7647)

clojure.lang.ExceptionInfo: Failed to load library {:libname "python3.6m", :paths [[:system ["python3.6m"]] [:java-library-path ["C\python3.6m.dll" "\opt\Python\Python3.8\lib\python3.6m.dll" "C\python3.6m.dll" "\opt\Python\Python3.8\lib\python3.6m.dll" "C\python3.6m.dll" "\opt\Python\Python3.8\lib\python3.6m.dll" "C\python3.6m.dll" "\opt\Python\Python3.8\lib\python3.6m.dll" "C\python3.6m.dll" "\opt\Python\Python3.8\lib\python3.6m.dll" "C\python3.6m.dll" "\opt\Python\Python3.8\lib\python3.6m.dll" "C\python3.6m.dll" "\opt\Python\Python3.8\lib\python3.6m.dll" "C\python3.6m.dll" < ommited many other path combinations with */ python3.6m.dll > ]]]}

at tech.jna.base$do_load_library.invokeStatic(base.clj:158)

at tech.jna.base$do_load_library.invoke(base.clj:123)

at tech.jna.base$load_library.invokeStatic(base.clj:178)

at tech.jna.base$load_library.invoke(base.clj:174)

at tech.jna$load_library.invokeStatic(jna.clj:87)

at tech.jna$load_library.invoke(jna.clj:85)

at libpython_clj.python.interpreter$setup_direct_mappingBANG.invokeStatic(interpreter.clj:430)

at libpython_clj.python.interpreter$setup_direct_mappingBANG.invoke(interpreter.clj:428)

at libpython_clj.python.interpreter$initializeBANG.invokeStatic(interpreter.clj:464)

at libpython_clj.python.interpreter$initializeBANG.doInvoke(interpreter.clj:434)

at clojure.lang.RestFn.invoke(RestFn.java:619)

at libpython_clj.python$initializeBANG.invokeStatic(python.clj:242)

at libpython_clj.python$initializeBANG.doInvoke(python.clj:229)

at clojure.lang.RestFn.invoke(RestFn.java:457)

at python.config$eval35333.invokeStatic(config.clj:5)

at python.config$eval35333.invoke(config.clj:5)

at clojure.lang.Compiler.eval(Compiler.java:7176)

at clojure.lang.Compiler.load(Compiler.java:7635)

at python.config$eval35321.invokeStatic(form-init16284440531770716747.clj:1)

at python.config$eval35321.invoke(form-init16284440531770716747.clj:1)

at clojure.lang.Compiler.eval(Compiler.java:7176)

at clojure.lang.Compiler.eval(Compiler.java:7131)

at clojure.core$eval.invokeStatic(core.clj:3214)

at clojure.core$eval.invoke(core.clj:3210)

at clojure.main$repl$read_eval_print9068$fn9071.invoke(main.clj:414)

at clojure.main$repl$read_eval_print__9068.invoke(main.clj:414)

at clojure.main$repl$fn__9077.invoke(main.clj:435)

at clojure.main$repl.invokeStatic(main.clj:435)

at clojure.main$repl.doInvoke(main.clj:345)

at clojure.lang.RestFn.invoke(RestFn.java:1523)

at nrepl.middleware.interruptible_eval$evaluate.invokeStatic(interruptible_eval.clj:79)

at nrepl.middleware.interruptible_eval$evaluate.invoke(interruptible_eval.clj:55)

at nrepl.middleware.interruptible_eval$interruptible_eval$fn34601$fn34605.invoke(interruptible_eval.clj:142)

at clojure.lang.AFn.run(AFn.java:22)

at nrepl.middleware.session$session_exec$main_loop34702$fn34706.invoke(session.clj:171)

at nrepl.middleware.session$session_exec$main_loop__34702.invoke(session.clj:170)

at clojure.lang.AFn.run(AFn.java:22)

at java.base/java.lang.Thread.run(Thread.java:835)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/clj-python/libpython-clj/issues/121, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACPJX476KGMEF4IWKDAVGBLSFXJKVANCNFSM4RLKGIIA .

[Goldritter]

No it is a 64-Bit System. I installed a 64-Bit Python 3.8 and used a 64-Bit Java JDK. Here is the output from (System/getProperties):

(System/getProperties) => {"sun.desktop" "windows", "awt.toolkit" "sun.awt.windows.WToolkit", "java.specification.version" "12", "sun.cpu.isalist" "amd64", "sun.jnu.encoding" "Cp1252", "clojure.debug" "false", "java.vm.vendor" "Oracle Corporation", "sun.arch.data.model" "64", "user.variant" "", "java.vendor.url" "https://java.oracle.com/", "user.timezone" "Europe/Berlin", "java.vm.specification.version" "12", "os.name" "Windows 10", "sun.java.launcher" "SUN_STANDARD", "user.country" "DE", "sun.boot.library.path" "C:\Program Files\Java\jdk-12.0.1\bin", "jdk.debug" "release", "sun.cpu.endian" "little", "user.language" "de", "java.specification.vendor" "Oracle Corporation", "java.version.date" "2019-04-16", "java.home" "C:\Program Files\Java\jdk-12.0.1", "file.separator" "\", "line.separator" "\r\n", "java.vm.specification.vendor" "Oracle Corporation", "java.specification.name" "Java Platform API Specification", "java.awt.graphicsenv" "sun.awt.Win32GraphicsEnvironment", "user.script" "", "sun.management.compiler" "HotSpot 64-Bit Tiered Compilers", "clojure.tools.loggin.factory" "clojure.tools.logging.impl/log4j2-factory", "java.runtime.version" "12.0.1+12", "path.separator" ";", "os.version" "10.0", "java.runtime.name" "OpenJDK Runtime Environment", "file.encoding" "Cp1252", "java.vm.name" "OpenJDK 64-Bit Server VM", "java.vendor.url.bug" "https://bugreport.java.com/bugreport/", "java.version" "12.0.1", "os.arch" "amd64" }

[sogaiu]

Is it possible that one might have more luck with JDK 11?

[sogaiu]

BTW, I just tried with python 3.8.5 with some success on Windows 10.

I did something that was slightly different and might translate to something like:

(py/initialize! :python-executable "C:/opt/Python/Python3.8"
                :library-path "C:/opt/Python/Python3.8/python38.dll")

Here my intention was to use the .dll file instead of the .lib file.

The file path to the .dll file is a bit different from the .lib here -- the .lib file lives under libs, while the .dll file lives in the same directory as the python .exe. I don't know where the .dll (if it exists) is in the original system, but FWIW.

Without specifying :library-path I didn't have any success in starting up.

FWIW, I tested with JDK 8.

[Goldritter]

BTW, I just tried with python 3.8.5 with some success on Windows 10.

I did something that was slightly different and might translate to something like:

(py/initialize! :python-executable "C:/opt/Python/Python3.8"
                :library-path "C:/opt/Python/Python3.8/python38.dll")

Here my intention was to use the .dll file instead of the .lib file.

The file path to the .dll file is a bit different from the .lib here -- the .lib file lives under libs, while the .dll file lives in the same directory as the python .exe. I don't know where the .dll (if it exists) is in the original system, but FWIW.

Without specifying :library-path I didn't have any success in starting up.

FWIW, I tested with JDK 8.

Thanks. This works also for me. I run a successful test with:

(require '[libpython-clj.require :refer [require-python]]) (require-python '[numpy :as np]) (def test-ary (np/array [[1 2][3 4]])) test-ary => nil => :ok => #'python.config/test-ary => [[1 2] [3 4]]

With other libraries there are problems like (require-python '[tensorflow :as tf]) "Execution error at libpython-clj.metadata/path->py-obj (metadata.clj:293). Failed to find module or class tensorflow" but this is an other problem.

It might be a good idea to update the documentation for the initialization if this is really only a Windows Problem.

[jjtolton]

@sogaiu FTW!