Getting started py/initialize! error in various versions and pythons

[pdenno]

This is almost certainly not a problem with the code but perhaps something with the documentation or something I've missed reading it. I'm just stated investigating this very promising code (2.000 and today's 2.003) on macos. I've tried both anaconda and vanilla python with pyenv, and Java 16 and 17. In all cases I'm hitting a problem involving jdk.incubator.foreign.MemoryAddress.

Here it is with conda, and python 3.7.11 with echo $LD_LIBRARY_PATH ==> /Users/pdenno/opt/anaconda3/envs/squid37/lib started with clj from a the shell prompt:

(ns my-py-clj.config
  (:require [libpython-clj2.python :as py]))

(py/initialize! :python-executable "/Users/pdenno/opt/anaconda3/envs/squid37/bin/python3.7"
                :library-path      "/Users/pdenno/opt/anaconda3/envs/squid37/lib/libpython3.7m.dylib")

INFO: Startup info 
{:lib-version "3.7", :java-library-path-addendum "/Users/pdenno/opt/anaconda3/envs/squid37/lib", :exec-prefix "/Users/pdenno/opt/anaconda3/envs/squid37", :executable "/Users/pdenno/opt/anaconda3/envs/squid37/bin/python3.7", :libnames ("/Users/pdenno/opt/anaconda3/envs/squid37/lib/libpython3.7m.dylib" "python3.7"), :prefix "/Users/pdenno/opt/anaconda3/envs/squid37", :base-prefix "/Users/pdenno/opt/anaconda3/envs/squid37", :libname "/Users/pdenno/opt/anaconda3/envs/squid37/lib/libpython3.7m.dylib", :base-exec-prefix "/Users/pdenno/opt/anaconda3/envs/squid37", :python-home "/Users/pdenno/opt/anaconda3/envs/squid37", :version [3 7 11], :platform "darwin"}
Oct 22, 2021 5:47:24 PM clojure.tools.logging$eval661$fn__664 invoke
INFO: Prefixing java library path: /Users/pdenno/opt/anaconda3/envs/squid37/lib
Oct 22, 2021 5:47:25 PM clojure.tools.logging$eval661$fn__664 invoke
INFO: Loading python library: /Users/pdenno/opt/anaconda3/envs/squid37/lib/libpython3.7m.dylib
Oct 22, 2021 5:47:26 PM clojure.tools.logging$eval661$fn__664 invoke
INFO: Reference thread starting
Oct 22, 2021 5:47:26 PM clojure.tools.logging$eval661$fn__664 invoke
INFO: Unable to find direct buffer constructor -
falling back to jdk16 memory model.
Oct 22, 2021 5:47:26 PM clojure.tools.logging$eval661$fn__664 invoke
SEVERE: Exception while releasing object
Syntax error compiling at (tech/v3/datatype/ffi/nio_buf_mmodel.clj:1:1).

...
Syntax error (ClassNotFoundException) compiling at (tech/v3/datatype/ffi/nio_buf_mmodel.clj:1:1).
jdk.incubator.foreign.MemoryAddress```

[cnuernber]

I just now have a fix for you today thanks to an offline email chain from another user running jdk-17 on mac m-1.

That error happens because various parts of the old system for taking a c pointer and making a nio buffer out of it used a class - sun.misc.Unsafe.

To enable the new way with jdk-17 check the deps.edn for libpython-clj has the answer:

:jvm-opts ["--add-modules" "jdk.incubator.foreign"
                       "--enable-native-access=ALL-UNNAMED"]}

This unlocks the foreign module which allows me to use various other tools to do the same operation - native pointer to nio buffer.

This fix means that libpython-clj works on jdk-8 through jdk-11 but then only on jdk-17 moving forward. jdk-16 no longer works.

[jjtolton]

Great work!

On Fri, Oct 22, 2021 at 6:26 PM Chris Nuernberger @.***> wrote:

I just now have a fix for you today thanks to an offline email chain from another user running jdk-17 on mac m-1.

That error happens because various parts of the old system for taking a c pointer and making a nio buffer out of it used a class - sun.misc.Unsafe.

To enable the new way with jdk-17 check the deps.edn https://github.com/clj-python/libpython-clj/blob/master/deps.edn#L10 for libpython-clj has the answer:

:jvm-opts ["--add-modules" "jdk.incubator.foreign" "--enable-native-access=ALL-UNNAMED"]}

This unlocks the foreign module which allows me to use various other tools to do the same operation - native pointer to nio buffer.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/clj-python/libpython-clj/issues/180#issuecomment-949993210, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACPJX47YBQBEEZAL2ATJV6DUIHQJZANCNFSM5GRPQSTQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[pdenno]

Works! Excellent! This opens up a whole new dimension to my clojure hacking. Thank you!