search

ninia / jep

Embed Python in Java
Other
1.27k stars 146 forks source link

(bug) Install on macOS arm64 comes with x86_64 native libraries #495

Closed Prong4107 closed 6 months ago

Prong4107 commented 10 months ago

Describe the problem When installing jep using a venv on a arm64 mac with pip, native libraries results are for the wrong CPU architecture, resulting in a crash when trying to load libjep.jnilib

**Environment

Hardware Overview:

  Model Name:   Mac mini
  Model Identifier: Macmini9,1
  Model Number: MGNR3LL/A
  Chip: Apple M1
  Total Number of Cores:    8 (4 performance and 4 efficiency)
  Memory:   8 GB
  System Firmware Version:  8422.141.2
  OS Loader Version:    8422.141.2
  Serial Number (system):   H2WDR39CQ6NV
  Hardware UUID:    9D7A7B24-F69B-5407-AB83-45C4115605B5
  Provisioning UDID:    00008103-000664A01840291E
  Activation Lock Status:   Disabled

Hardware Overview:

  Model Name:   Mac mini
  Model Identifier: Macmini9,1
  Model Number: MGNR3LL/A
  Chip: Apple M1
  Total Number of Cores:    8 (4 performance and 4 efficiency)
  Memory:   8 GB
  System Firmware Version:  8422.141.2
  OS Loader Version:    8422.141.2
  Serial Number (system):   H2WDR39CQ6NV
  Hardware UUID:    9D7A7B24-F69B-5407-AB83-45C4115605B5
  Provisioning UDID:    00008103-000664A01840291E
  Activation Lock Status:   Disabled
$ uname -a
Darwin EU852 22.6.0 Darwin Kernel Version 22.6.0: Wed Jul  5 22:22:52 PDT 2023; root:xnu-8796.141.3~6/RELEASE_ARM64_T8103 arm64

Python 3.11.5 installed with homebrew

$ java --version
openjdk 11.0.16.1 2022-08-12 LTS
OpenJDK Runtime Environment Microsoft-40648 (build 11.0.16.1+1-LTS)
OpenJDK 64-Bit Server VM Microsoft-40648 (build 11.0.16.1+1-LTS, mixed mode)

Reproduction

$ python3 --version
Python 3.11.5

$ python3 -m venv venv

$ source venv/bin/activate

(venv) $ pip install jep
Collecting jep
  Using cached jep-4.1.1-cp311-cp311-macosx_13_0_arm64.whl
Installing collected packages: jep
Successfully installed jep-4.1.1

(venv) $ file venv/lib/python3.11/site-packages/jep/libjep.jnilib 
venv/lib/python3.11/site-packages/jep/libjep.jnilib: Mach-O 64-bit bundle x86_64

(venv) $ file venv/lib/python3.11/site-packages/jep/

(venv) $ file venv/lib/python3.11/site-packages/jep/jep.cpython-311-darwin.so 
venv/lib/python3.11/site-packages/jep/jep.cpython-311-darwin.so: Mach-O 64-bit bundle x86_64

(venv) $ jep
java.lang.UnsatisfiedLinkError: Can't load library: /Users/user934234/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/jep/libjep.jnilib
    at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2633)
    at java.base/java.lang.Runtime.load0(Runtime.java:768)
    at java.base/java.lang.System.load(System.java:1837)
    at jep.LibraryLocator.searchPackageDir(LibraryLocator.java:243)
    at jep.LibraryLocator.searchSitePackages(LibraryLocator.java:146)
    at jep.LibraryLocator.findJepLibrary(LibraryLocator.java:325)
    at jep.MainInterpreter.initialize(MainInterpreter.java:131)
    at jep.MainInterpreter.getMainInterpreter(MainInterpreter.java:101)
    at jep.Jep.<init>(Jep.java:133)
    at jep.SharedInterpreter.<init>(SharedInterpreter.java:60)
    at jep.Run.run(Run.java:49)
    at jep.Run.main(Run.java:146)
bsteffensmeier commented 10 months ago

Please ensure that your Java and Python are both using the same architecture. The jep library is built to match your python architecture so it appears you are mixing an x86_64 python with an arm64 Java which is not supported.

If you are having trouble installing compatible versions I recommend reviewing the previous discussion in #379, #389, #423, and #442.

Prong4107 commented 10 months ago

Actually, it seems my problem was that I installed jep in x86_64 mode a first time, and pip cached the build, so when I installed jep in a venv a second time in arm64, it used the cached x86_64 binaries instead of rebuilding for arm64.

Running pip cache purge before pip install seems to have solved it.

Not entirely sure how the inconsistencies in architectures first happened though.

Thanks for the help.