search

clj-python / libpython-clj

Python bindings for Clojure
Eclipse Public License 2.0
1.06k stars 69 forks source link

namespaces not found when using require macro #141

Closed miloscu closed 3 years ago

miloscu commented 3 years ago

consider my code

(defproject manualpy "0.1.0-SNAPSHOT"
  :dependencies [[org.clojure/clojure "1.10.1"]
                 [clj-python/libpython-clj "2.00-alpha-6"]])
;;src/manualpy/core.clj
(ns manualpy.core
  (:require [libpython-clj.python :as py :refer [py. py.. py.-]]
            [libpython-clj.require :refer [require-python]]
            ))

(py/initialize!
 :python-executable "C:/tools/anaconda3/envs/cljenv1/python.exe"
 :library-path "C:/tools/anaconda3/envs/cljenv1/python36.dll"
 :windows-anaconda-activate-bat "C:/tools/Anaconda3/Scripts/activate.bat")

I get the following syntax error:

; Syntax error compiling at (libpython_clj\require.clj:1:1).
; namespace 'libpython-clj.metadata' not found

this disallows me to do things like

(require-python '[math :as pmath])
(pmath/sin 1.0)

; Syntax error compiling at (c:\clojure-projects\manualpy\src\manualpy\core.clj:17:1).
; Unable to resolve symbol: require-python in this context

this works fine

(def np (py/import-module "numpy"))
(def onesary (py/call-attr np "ones" [2 3]))
onesary

Can someone help me resolve the issue? I'm not sure what I'm doing wrong

cnuernber commented 3 years ago

Could you go into the metadata namespace and just compile that file alone in your repl? When a dependent namespace 'isn't found' a lot of times that is due to a compilation error that is getting swallowed.

jjtolton commented 3 years ago

Also make sure there isn’t anything named “metadata” or “libpython_clj” possibly confusing your class path!

On Wed, Dec 9, 2020 at 3:34 PM Chris Nuernberger notifications@github.com wrote:

Could you go into the metadata namespace and just compile that file alone in your repl? When a dependent namespace 'isn't found' a lot of times that is due to a compilation error that is getting swalloed.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/clj-python/libpython-clj/issues/141#issuecomment-742032059, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACPJX45XK4MQEESM5T34GALST7NLRANCNFSM4USYJPFQ .

charignon commented 3 years ago

I have the exact same problem with a pretty minimal example, do you have suggestions on how to find the conflict?

cnuernber commented 3 years ago

in a repl attempt to compile the metadata namespace. That should produce an error that is getting swallowed.

glittershark commented 3 years ago

This is happening for me as well, here's the error I'm getting:

Syntax error (IllegalStateException) compiling at (libpython_clj/metadata.clj:13:1).
Failed to find constructor f java.nio.DirectByteBuffer: $s
glittershark commented 3 years ago

and here's the full exception:

#error {
 :cause "Failed to find constructor f java.nio.DirectByteBuffer: $s"
 :via
 [{:type clojure.lang.Compiler$CompilerException
   :message "Syntax error compiling at (libpython_clj/metadata.clj:13:1)."
   :data #:clojure.error{:phase :compile-syntax-check, :line 13, :column 1, :source "libpython_clj/metadata.clj"}
   :at [clojure.lang.Compiler load "Compiler.java" 7650]}
  {:type java.lang.ExceptionInInitializerError
   :message nil
   :at [tech.v3.datatype.native_buffer$unsafe invokeStatic "native_buffer.clj" 28]}
  {:type java.lang.IllegalStateException
   :message "Failed to find constructor f java.nio.DirectByteBuffer: $s"
   :at [xerial.larray.buffer.UnsafeUtil findDirectByteBufferConstructor "UnsafeUtil.java" 40]}]
 :trace
 [[xerial.larray.buffer.UnsafeUtil findDirectByteBufferConstructor "UnsafeUtil.java" 40]
  [xerial.larray.buffer.UnsafeUtil <clinit> "UnsafeUtil.java" 44]
  [tech.v3.datatype.native_buffer$unsafe invokeStatic "native_buffer.clj" 28]
  [tech.v3.datatype.native_buffer$unsafe invoke "native_buffer.clj" 28]
  [libpython_clj.python.interop$register_type_BANG_$fn__10295 invoke "interop.clj" 227]
  [clojure.lang.AFn applyToHelper "AFn.java" 152]
  [clojure.lang.AFn applyTo "AFn.java" 144]
  [clojure.core$apply invokeStatic "core.clj" 665]
  [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1975]
  [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1975]
  [clojure.lang.RestFn invoke "RestFn.java" 425]
  [libpython_clj.python.interop$register_type_BANG_ invokeStatic "interop.clj" 205]
  [libpython_clj.python.interop$register_type_BANG_ invoke "interop.clj" 188]
  [libpython_clj.python.interop$register_bridge_type_BANG_ invokeStatic "interop.clj" 282]
  [libpython_clj.python.interop$register_bridge_type_BANG_ doInvoke "interop.clj" 275]
  [clojure.lang.RestFn invoke "RestFn.java" 397]
  [libpython_clj.python$initialize_BANG_ invokeStatic "python.clj" 248]
  [libpython_clj.python$initialize_BANG_ doInvoke "python.clj" 230]
  [clojure.lang.RestFn invoke "RestFn.java" 397]
  [libpython_clj.metadata$eval16351 invokeStatic "metadata.clj" 13]
  [libpython_clj.metadata$eval16351 invoke "metadata.clj" 13]
  [clojure.lang.Compiler eval "Compiler.java" 7179]
  [clojure.lang.Compiler load "Compiler.java" 7638]
  [clojure.lang.RT loadResourceScript "RT.java" 381]
  [clojure.lang.RT loadResourceScript "RT.java" 372]
  [clojure.lang.RT load "RT.java" 459]
  [clojure.lang.RT load "RT.java" 424]
  [clojure.core$load$fn__6848 invoke "core.clj" 6129]
  [clojure.core$load invokeStatic "core.clj" 6128]
  [clojure.core$load doInvoke "core.clj" 6112]
  [clojure.lang.RestFn invoke "RestFn.java" 408]
  [clojure.core$load_one invokeStatic "core.clj" 5911]
  [clojure.core$load_one invoke "core.clj" 5906]
  [clojure.core$load_lib$fn__6788 invoke "core.clj" 5951]
  [clojure.core$load_lib invokeStatic "core.clj" 5950]
  [clojure.core$load_lib doInvoke "core.clj" 5931]
  [clojure.lang.RestFn applyTo "RestFn.java" 142]
  [clojure.core$apply invokeStatic "core.clj" 667]
  [clojure.core$load_libs invokeStatic "core.clj" 5988]
  [clojure.core$load_libs doInvoke "core.clj" 5972]
  [clojure.lang.RestFn applyTo "RestFn.java" 137]
  [clojure.core$apply invokeStatic "core.clj" 667]
  [clojure.core$require invokeStatic "core.clj" 6010]
  [clojure.core$require doInvoke "core.clj" 6010]
  [clojure.lang.RestFn invoke "RestFn.java" 408]
  [user$eval5 invokeStatic "NO_SOURCE_FILE" 1]
  [user$eval5 invoke "NO_SOURCE_FILE" 1]
  [clojure.lang.Compiler eval "Compiler.java" 7179]
  [clojure.lang.Compiler eval "Compiler.java" 7134]
  [clojure.core$eval invokeStatic "core.clj" 3216]
  [clojure.core$eval invoke "core.clj" 3212]
  [clojure.main$repl$read_eval_print__9100$fn__9103 invoke "main.clj" 437]
  [clojure.main$repl$read_eval_print__9100 invoke "main.clj" 437]
  [clojure.main$repl$fn__9109 invoke "main.clj" 458]
  [clojure.main$repl invokeStatic "main.clj" 458]
  [clojure.main$repl_opt invokeStatic "main.clj" 522]
  [clojure.main$main invokeStatic "main.clj" 667]
  [clojure.main$main doInvoke "main.clj" 616]
  [clojure.lang.RestFn invoke "RestFn.java" 397]
  [clojure.lang.AFn applyToHelper "AFn.java" 152]
  [clojure.lang.RestFn applyTo "RestFn.java" 132]
  [clojure.lang.Var applyTo "Var.java" 705]
  [clojure.main main "main.java" 40]]}
cnuernber commented 3 years ago

That specific error is related to the java version. Versions 14+ do not include a constructor that we use in order to create a direct byte buffer from a long integer.

JNA does include this pathway, however, and I guess when JNA is loaded into the process I should use JNA to do this operation. That would allow JDK14+ support for some time.

cnuernber commented 3 years ago

I think I have a fix for this for now.

cnuernber commented 3 years ago

Could you try libpython-clj version 2.00-alpha-7? This should work with JDK14+, at least it does on my machine.

glittershark commented 3 years ago

That works! I do get the following warning though:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by tech.v3.datatype.UnsafeUtil (file:/home/grfn/.m2/repository/cnuernber/dtype-next/6.00-beta-11/dtype-next-6.00-beta-11.jar) to constructor java.nio.DirectByteBuffer(long,int)
WARNING: Please consider reporting this to the maintainers of tech.v3.datatype.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
cnuernber commented 3 years ago

Yep, that is just what it is for now. The only other option is to make the base dtype-next library dependent (again) upon JNA so I get an ptr,length->byte buffer constructor that isn't jvm-version dependent.

Baby steps, glad this works for you :-). Leaving open for just a short while then closing if no one objects.

cnuernber commented 3 years ago

@miloscu - Does the newest version work for you?

cnuernber commented 3 years ago

Closing as I believe this was a JDK14 issue.