search

clj-python / libpython-clj

Python bindings for Clojure
Eclipse Public License 2.0
1.05k stars 68 forks source link

Error requiring libpython-clj2.require on M1 Mac - JDK-18 #204

Closed Hendekagon closed 1 year ago

Hendekagon commented 2 years ago

I'm using libpython-clj 2.018 with aarch64 jdk-18.0.1.jdk, with :jdk-17 alias' :jvm-opts and I get this error when (require '[libpython-clj2.require :refer [require-python]]) on an M1 Mac:

INFO: Unable to find direct buffer constructor - falling back to jdk16 memory model. Reflection warning, tech/v3/datatype/ffi/nio_buf_mmodel.clj:14:22 - call to method asSegment on jdk.incubator.foreign.MemoryAddress can't be resolved (no such method). Reflection warning, tech/v3/datatype/ffi/nio_buf_mmodel.clj:15:22 - reference to field asByteBuffer can't be resolved. Reflection warning, tech/v3/datatype/ffi/nio_buf_mmodel.clj:19:22 - call to method order can't be resolved (target class is unknown). Reflection warning, tech/v3/datatype/ffi/nio_buf_mmodel.clj:20:19 - call to method order can't be resolved (target class is unknown). Apr 26, 2022 10:12:23 AM libpython-clj2.python.ffi invoke SEVERE: Exception while releasing object java.lang.IllegalArgumentException: No matching method asSegment found taking 2 args for class jdk.internal.foreign.MemoryAddressImpl at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:127) at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:102) at tech.v3.datatype.ffi.nio_buf_mmodel$direct_buffer_constructor.invokeStatic(nio_buf_mmodel.clj:14) at tech.v3.datatype.ffi.nio_buf_mmodel$direct_buffer_constructor.invoke(nio_buf_mmodel.clj:10)...

what am I doing wrong ?

jjtolton commented 2 years ago

Can you paste the full stacktrace? I only see warnings so far. Does it crash?

Hendekagon commented 2 years ago

here:

WARNING: Using incubator modules: jdk.incubator.foreign, jdk.incubator.vector
  Clojure 1.11.1

  (require '[libpython-clj2.require :as pyr :refer [require-python]])
  WARNING: abs already refers to: #'clojure.core/abs in namespace: tech.v3.datatype.functional-api, being replaced by: #'tech.v3.datatype.functional-api/abs
  WARNING: infinite? already refers to: #'clojure.core/infinite? in namespace: tech.v3.datatype.functional-api, being replaced by: #'tech.v3.datatype.functional-api/infinite?
  WARNING: abs already refers to: #'clojure.core/abs in namespace: tech.v3.datatype.functional, being replaced by: #'tech.v3.datatype.functional/abs
  WARNING: infinite? already refers to: #'clojure.core/infinite? in namespace: tech.v3.datatype.functional, being replaced by: #'tech.v3.datatype.functional/infinite?
  Apr 26, 2022 11:43:54 AM libpython-clj2.python.info invoke
  INFO: Detecting startup info
  Apr 26, 2022 11:43:54 AM libpython-clj2.python invoke
  INFO: Startup info {:lib-version "3.8", :java-library-path-addendum "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib", :exec-prefix "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8", :executable "/Library/Developer/CommandLineTools/usr/bin/python3", :libnames ("python3.8m" "python3.8"), :prefix "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8", :base-prefix "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8", :libname "python3.8m", :base-exec-prefix "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8", :python-home "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8", :version [3 8 9], :platform "darwin"}
  Apr 26, 2022 11:43:54 AM libpython-clj2.python invoke
  INFO: Prefixing java library path: /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib
  Apr 26, 2022 11:43:55 AM libpython-clj2.python invoke
  INFO: Loading python library: /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/libpython3.8.dylib
  Apr 26, 2022 11:43:55 AM tech.v3.resource.gc invoke
  INFO: Reference thread starting
  Apr 26, 2022 11:43:55 AM tech.v3.datatype.nio-buffer invoke
  INFO: Unable to find direct buffer constructor -
  falling back to jdk16 memory model.
  Reflection warning, tech/v3/datatype/ffi/nio_buf_mmodel.clj:14:22 - call to method asSegment on jdk.incubator.foreign.MemoryAddress can't be resolved (no such method).
  Reflection warning, tech/v3/datatype/ffi/nio_buf_mmodel.clj:15:22 - reference to field asByteBuffer can't be resolved.
  Reflection warning, tech/v3/datatype/ffi/nio_buf_mmodel.clj:19:22 - call to method order can't be resolved (target class is unknown).
  Reflection warning, tech/v3/datatype/ffi/nio_buf_mmodel.clj:20:19 - call to method order can't be resolved (target class is unknown).
  Apr 26, 2022 11:43:55 AM libpython-clj2.python.ffi invoke
  SEVERE: Exception while releasing object
  java.lang.IllegalArgumentException: No matching method asSegment found taking 2 args for class jdk.internal.foreign.MemoryAddressImpl
  at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:127)
  at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:102)
  at tech.v3.datatype.ffi.nio_buf_mmodel$direct_buffer_constructor.invokeStatic(nio_buf_mmodel.clj:14)
  at tech.v3.datatype.ffi.nio_buf_mmodel$direct_buffer_constructor.invoke(nio_buf_mmodel.clj:10)
  at clojure.lang.Var.invoke(Var.java:399)
  at tech.v3.datatype.nio_buffer$native_buf__GT_nio_buf.invokeStatic(nio_buffer.clj:113)
  at tech.v3.datatype.nio_buffer$native_buf__GT_nio_buf.invoke(nio_buffer.clj:108)
  at libpython_clj2.python.ffi$pystr__GT_str.invokeStatic(ffi.clj:785)
  at libpython_clj2.python.ffi$pystr__GT_str.invoke(ffi.clj:773)
  at libpython_clj2.python.ffi$pytype_name.invokeStatic(ffi.clj:793)
  at libpython_clj2.python.ffi$pytype_name.invoke(ffi.clj:789)
  at libpython_clj2.python.ffi$pyobject_type_kwd$reify__10514.apply(ffi.clj:809)
  at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708)
  at libpython_clj2.python.ffi$pyobject_type_kwd.invokeStatic(ffi.clj:805)
  at libpython_clj2.python.ffi$pyobject_type_kwd.invoke(ffi.clj:802)
  at libpython_clj2.python.ffi$wrap_obj_ptr$fn__10517.invoke(ffi.clj:840)
  at libpython_clj2.python.gc$track$fn__9936.invoke(gc.clj:37)
  at tech.resource.GCReference.run(GCReference.java:25)
  at libpython_clj2.python.gc$clear_stack_context.invokeStatic(gc.clj:59)
  at libpython_clj2.python.gc$clear_stack_context.invoke(gc.clj:56)
  at libpython_clj2.python.io_redirect$setup_std_writer$fn__12372.invoke(io_redirect.clj:44)
  at clojure.lang.AFn.applyToHelper(AFn.java:152)
  at clojure.lang.AFn.applyTo(AFn.java:144)
  at clojure.core$apply.invokeStatic(core.clj:667)
  at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1990)
  at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1990)
  at clojure.lang.RestFn.invoke(RestFn.java:425)
  at libpython_clj2.python.io_redirect$setup_std_writer.invokeStatic(io_redirect.clj:44)
  at libpython_clj2.python.io_redirect$setup_std_writer.invoke(io_redirect.clj:40)
  at libpython_clj2.python.io_redirect$redirect_io_BANG_.invokeStatic(io_redirect.clj:54)
  at libpython_clj2.python.io_redirect$redirect_io_BANG_.invoke(io_redirect.clj:52)
  at libpython_clj2.python$initialize_BANG_$fn__12403.invoke(python.clj:107)
  at libpython_clj2.python$initialize_BANG_.invokeStatic(python.clj:100)
  at libpython_clj2.python$initialize_BANG_.doInvoke(python.clj:46)
  at clojure.lang.RestFn.invoke(RestFn.java:397)
  at libpython_clj2.metadata$eval12678.invokeStatic(metadata.clj:17)
  at libpython_clj2.metadata$eval12678.invoke(metadata.clj:17)
  at clojure.lang.Compiler.eval(Compiler.java:7194)
  Execution error (IllegalArgumentException) at tech.v3.datatype.ffi.nio-buf-mmodel/direct-buffer-constructor (nio_buf_mmodel.clj:14).
  No matching method asSegment found taking 2 args for class jdk.internal.foreign.MemoryAddressImpl
  at clojure.lang.Compiler.load(Compiler.java:7653)
  at clojure.lang.RT.loadResourceScript(RT.java:381)
  at clojure.lang.RT.loadResourceScript(RT.java:372)
  at clojure.lang.RT.load(RT.java:459)
  at clojure.lang.RT.load(RT.java:424)
  at clojure.core$load$fn__6908.invoke(core.clj:6161)
  at clojure.core$load.invokeStatic(core.clj:6160)
  at clojure.core$load.doInvoke(core.clj:6144)
  at clojure.lang.RestFn.invoke(RestFn.java:408)
  at clojure.core$load_one.invokeStatic(core.clj:5933)
  at clojure.core$load_one.invoke(core.clj:5928)
  at clojure.core$load_lib$fn__6850.invoke(core.clj:5975)
  at clojure.core$load_lib.invokeStatic(core.clj:5974)
  at clojure.core$load_lib.doInvoke(core.clj:5953)
  at clojure.lang.RestFn.applyTo(RestFn.java:142)
  at clojure.core$apply.invokeStatic(core.clj:669)
  at clojure.core$load_libs.invokeStatic(core.clj:6016)
  at clojure.core$load_libs.doInvoke(core.clj:6000)
  at clojure.lang.RestFn.applyTo(RestFn.java:137)
  at clojure.core$apply.invokeStatic(core.clj:669)
  at clojure.core$require.invokeStatic(core.clj:6038)
  at clojure.core$require.doInvoke(core.clj:6038)
  at clojure.lang.RestFn.invoke(RestFn.java:482)
  at libpython_clj2.require$eval1932$loading__6789__auto____1933.invoke(require.clj:1)
  at libpython_clj2.require$eval1932.invokeStatic(require.clj:1)
  at libpython_clj2.require$eval1932.invoke(require.clj:1)
  at clojure.lang.Compiler.eval(Compiler.java:7194)
  at clojure.lang.Compiler.eval(Compiler.java:7183)
  at clojure.lang.Compiler.load(Compiler.java:7653)
  at clojure.lang.RT.loadResourceScript(RT.java:381)
  at clojure.lang.RT.loadResourceScript(RT.java:372)
  at clojure.lang.RT.load(RT.java:459)
  at clojure.lang.RT.load(RT.java:424)
  at clojure.core$load$fn__6908.invoke(core.clj:6161)
  at clojure.core$load.invokeStatic(core.clj:6160)
  at clojure.core$load.doInvoke(core.clj:6144)
  at clojure.lang.RestFn.invoke(RestFn.java:408)
  at clojure.core$load_one.invokeStatic(core.clj:5933)
  at clojure.core$load_one.invoke(core.clj:5928)
  at clojure.core$load_lib$fn__6850.invoke(core.clj:5975)
  at clojure.core$load_lib.invokeStatic(core.clj:5974)
  at clojure.core$load_lib.doInvoke(core.clj:5953)
  at clojure.lang.RestFn.applyTo(RestFn.java:142)
  at clojure.core$apply.invokeStatic(core.clj:669)
  at clojure.core$load_libs.invokeStatic(core.clj:6016)
  at clojure.core$load_libs.doInvoke(core.clj:6000)
  at clojure.lang.RestFn.applyTo(RestFn.java:137)
  at clojure.core$apply.invokeStatic(core.clj:669)
  at clojure.core$require.invokeStatic(core.clj:6038)
  at clojure.core$require.doInvoke(core.clj:6038)
  at clojure.lang.RestFn.invoke(RestFn.java:408)
  at test.test$eval1926.invokeStatic(test.clj:5)
  at test.test$eval1926.invoke(test.clj:5)
  at clojure.lang.Compiler.eval(Compiler.java:7194)
  at clojure.lang.Compiler.eval(Compiler.java:7149)
  at clojure.core$eval.invokeStatic(core.clj:3215)
  at clojure.core$eval.invoke(core.clj:3211)
  at nrepl.middleware.interruptible_eval$evaluate$fn__968$fn__969.invoke(interruptible_eval.clj:87)
  at clojure.lang.AFn.applyToHelper(AFn.java:152)
  at clojure.lang.AFn.applyTo(AFn.java:144)
  at clojure.core$apply.invokeStatic(core.clj:667)
  at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1990)
  at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1990)
  at clojure.lang.RestFn.invoke(RestFn.java:425)
  at nrepl.middleware.interruptible_eval$evaluate$fn__968.invoke(interruptible_eval.clj:87)
  at clojure.main$repl$read_eval_print__9206$fn__9209.invoke(main.clj:437)
  at clojure.main$repl$read_eval_print__9206.invoke(main.clj:437)
  at clojure.main$repl$fn__9215.invoke(main.clj:458)
  at clojure.main$repl.invokeStatic(main.clj:458)
  at clojure.main$repl.doInvoke(main.clj:368)
  at clojure.lang.RestFn.invoke(RestFn.java:1523)
  at nrepl.middleware.interruptible_eval$evaluate.invokeStatic(interruptible_eval.clj:84)
  at nrepl.middleware.interruptible_eval$evaluate.invoke(interruptible_eval.clj:56)
  at nrepl.middleware.interruptible_eval$interruptible_eval$fn__999$fn__1003.invoke(interruptible_eval.clj:152)
  at clojure.lang.AFn.run(AFn.java:22)
  at nrepl.middleware.session$session_exec$main_loop__1067$fn__1071.invoke(session.clj:202)
  at nrepl.middleware.session$session_exec$main_loop__1067.invoke(session.clj:201)
  at clojure.lang.AFn.run(AFn.java:22)
  at java.base/java.lang.Thread.run(Thread.java:833)
cnuernber commented 2 years ago

This is a JDK issue. The foreign interface changed again on JDK-18 and I haven't updated dtype-next to handle it. JDK-17 will work iwth your setup I believe.

Hendekagon commented 2 years ago

thank you! that was it I still get this

INFO: Unable to find direct buffer constructor -
falling back to jdk16 memory model.

but otherwise it now works

cnuernber commented 2 years ago

OK, let's leave this open. I won't fix this for JDK-18 as JDK-19 is coming quite soon and we will need to see what API changes it includes.

wbridewell commented 1 year ago

With JDK-19 released and JDK-20 in early access, has java.lang.foreign stabilized enough to revisit an FFI update?

cnuernber commented 1 year ago

I just landed a PR in dtype-next that purports to do just that :-). I haven't tested it yet.

cnuernber commented 1 year ago

JDK-19 is now supported out of the box with latest libpython-clj :-).