search

ninia / jep

Embed Python in Java
Other
1.3k stars 147 forks source link

OSX 12.7.3 M1 Using JEP Getting Started Gets "Unstatisfied Link Error.....incompatible architecture (have (arm64), need (x86_64))" #544

Closed johnferguson-at-nuix closed 1 month ago

johnferguson-at-nuix commented 1 month ago

Describe the problem

  1. Installed JEP successfully (on two different python versions using pyenv)
  2. Trying to run example code from https://github.com/ninia/jep/wiki/Getting-Started#example-code. Hello World
  3. Run code from within IntelliJ
  4. Set -Djava.library.path=/Users/johnferguson/.pyenv/versions/3.10.12/lib/python3.10/site-packages/jep because the shared library is there? Instructions in tutorial mentioned this.
     Exception in thread "main" java.lang.UnsatisfiedLinkError: /Users/johnferguson/.pyenv/versions/3.10.12/lib/python3.10/site-packages/jep/libjep.jnilib: dlopen(/Users/johnferguson/.pyenv/versions/3.10.12/lib/python3.10/site-packages/jep/libjep.jnilib, 0x0001): tried: '/Users/johnferguson/.pyenv/versions/3.10.12/lib/python3.10/site-packages/jep/libjep.jnilib' (mach-o file, but is an incompatible architecture (have (arm64), need (x86_64)))
     at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
    at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:388)
    at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:232)
    at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:174)
    at java.base/jdk.internal.loader.NativeLibraries.findFromPaths(NativeLibraries.java:321)
    at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:287)
    at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2427)
    at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:818)
    at java.base/java.lang.System.loadLibrary(System.java:1989)
    at jep.MainInterpreter.initialize(MainInterpreter.java:129)
    at jep.MainInterpreter.getMainInterpreter(MainInterpreter.java:101)
    at jep.Jep.<init>(Jep.java:133)
    at jep.SharedInterpreter.<init>(SharedInterpreter.java:60)
    at com.nuix.load.slacker.Application.main(Application.java:20)

Extra bits of information

On another OSX 14.4.1 M3, Rosetta, Java 21 Termurin, pyenv Python system able to run Java Application from IntelliJ using JEP and Hugging Face pipelines.

On the machine with the issue when I try this command:

 ~/.pyenv/shims/jep

I get 

java.lang.UnsatisfiedLinkError: /Users/johnferguson/.pyenv/versions/3.12.2/lib/python3.12/site-packages/jep/libjep.jnilib: dlopen(/Users/johnferguson/.pyenv/versions/3.12.2/lib/python3.12/site-packages/jep/libjep.jnilib, 0x0001): tried: '/Users/johnferguson/.pyenv/versions/3.12.2/lib/python3.12/site-packages/jep/libjep.jnilib' (mach-o file, but is an incompatible architecture (have (arm64), need (x86_64)))
    at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
    at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:388)
    at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:232)
    at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:174)
    at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2394)
    at java.base/java.lang.Runtime.load0(Runtime.java:755)
    at java.base/java.lang.System.load(System.java:1953)
    at jep.LibraryLocator.searchPackageDir(LibraryLocator.java:243)
    at jep.LibraryLocator.searchSitePackages(LibraryLocator.java:146)
    at jep.LibraryLocator.findJepLibrary(LibraryLocator.java:325)
    at jep.MainInterpreter.initialize(MainInterpreter.java:131)
    at jep.MainInterpreter.getMainInterpreter(MainInterpreter.java:101)
    at jep.Jep.<init>(Jep.java:133)
    at jep.SharedInterpreter.<init>(SharedInterpreter.java:60)
    at jep.Run.run(Run.java:49)
    at jep.Run.main(Run.java:146)

When I run this pyenv shim on the good machine I get a python prompt.

Also, I am using JEP 4.2.0 in both my pom and my pip

        <dependency>
            <groupId>black.ninia</groupId>
            <artifactId>jep</artifactId>
            <version>4.2.0</version>
        </dependency>

Environment (please complete the following information):

johnferguson-at-nuix commented 1 month ago

Also, the install of JEP after uninstalling and re-installing pyenv and pythons showed this:

 pip install jep
Collecting jep
  Using cached jep-4.2.0-cp312-cp312-macosx_12_0_arm64.whl
Installing collected packages: jep
Successfully installed jep-4.2.0
johnferguson-at-nuix commented 1 month ago

Okay so after some more reading of the documentation, and searching I found the aarch64 and x86_64 chaos was due to the fact that my work machine had jenv pointing to a x86_64 java distro used by my employer's main product. I am using aarch64 with Python and found https://github.com/ninia/jep/issues/379#issuecomment-1057057276 which clearly stated this.

Also, for those using pyenv - the warnings about using the same version of python that you are using JEP for (3.12.2) works but if I try 3.12.4 I got issues that manifest this kind of error:

Exception in thread "main" java.lang.UnsatisfiedLinkError: /Users/johnferguson/.pyenv/versions/3.12.4/lib/python3.12/site-packages/jep/libjep.jnilib: dlopen(/Users/johnferguson/.pyenv/versions/3.12.4/lib/python3.12/site-packages/jep/libjep.jnilib, 0x0001): Library not loaded: '/Users/johnferguson/.pyenv/versions/3.12.2/lib/libpython3.12.dylib'
  Referenced from: '/Users/johnferguson/.pyenv/versions/3.12.4/lib/python3.12/site-packages/jep/libjep.jnilib'
  Reason: tried: '/Users/johnferguson/.pyenv/versions/3.12.2/lib/libpython3.12.dylib' (no such file), '/usr/lib/libpython3.12.dylib' (no such file)