search

JuliaPy / PyCall.jl

Package to call Python functions from the Julia language
MIT License
1.47k stars 187 forks source link

"ImportError: no module named site" when using Mac M1 binary #925

Closed valentinsulzer closed 2 years ago

valentinsulzer commented 2 years ago

PyCall works with Julia 1.6:

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.0 (2021-03-24)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> versioninfo()
Julia Version 1.6.0
Commit f9720dc2eb (2021-03-24 12:55 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.6.0)
  CPU: Apple M1
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, westmere)

julia> using PyCall

But fails with Julia 1.7 (Mac M1 version)


               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.7.0-rc1 (2021-09-12)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> versioninfo()
Julia Version 1.7.0-rc1
Commit 9eade6195e (2021-09-12 06:45 UTC)
Platform Info:
  OS: macOS (arm64-apple-darwin20.5.0)
  CPU: Apple M1
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, cyclone)

julia> using PyCall
[ Info: Precompiling PyCall [438e738f-606a-5dbb-bf0a-cddfbfd45ab0]
ERROR: LoadError: could not load library "/usr/local/Caskroom/miniforge/base/envs/conda_jl/lib/libpython3.8.dylib"
dlopen(/usr/local/Caskroom/miniforge/base/envs/conda_jl/lib/libpython3.8.dylib, 9): no suitable image found.  Did find:
    /usr/local/Caskroom/miniforge/base/envs/conda_jl/lib/libpython3.8.dylib: mach-o, but wrong architecture
    /usr/local/Caskroom/miniforge/base/envs/conda_jl/lib/libpython3.8.dylib: mach-o, but wrong architecture. Please run `Pkg.build("PyCall")` if your Python build has changed
Stacktrace:
  [1] error(::String, ::String)
    @ Base ./error.jl:42
  [2] top-level scope
    @ ~/.julia/packages/PyCall/BD546/src/startup.jl:51
  [3] include(mod::Module, _path::String)
    @ Base ./Base.jl:420
  [4] include(x::String)
    @ PyCall ~/.julia/packages/PyCall/BD546/src/PyCall.jl:1
  [5] top-level scope
    @ ~/.julia/packages/PyCall/BD546/src/PyCall.jl:38
  [6] include
    @ ./Base.jl:420 [inlined]
  [7] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1318
  [8] top-level scope
    @ none:1
  [9] eval
    @ ./boot.jl:373 [inlined]
 [10] eval(x::Expr)
    @ Base.MainInclude ./client.jl:453
 [11] top-level scope
    @ none:1
in expression starting at /Users/vsulzer/.julia/packages/PyCall/BD546/src/startup.jl:41
in expression starting at /Users/vsulzer/.julia/packages/PyCall/BD546/src/PyCall.jl:1

caused by: could not load library "/usr/local/Caskroom/miniforge/base/envs/conda_jl/lib/libpython3.8.dylib"
dlopen(/usr/local/Caskroom/miniforge/base/envs/conda_jl/lib/libpython3.8.dylib, 9): no suitable image found.  Did find:
    /usr/local/Caskroom/miniforge/base/envs/conda_jl/lib/libpython3.8.dylib: mach-o, but wrong architecture
    /usr/local/Caskroom/miniforge/base/envs/conda_jl/lib/libpython3.8.dylib: mach-o, but wrong architecture
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:117
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:117
  [3] top-level scope
    @ ~/.julia/packages/PyCall/BD546/src/startup.jl:48
  [4] include(mod::Module, _path::String)
    @ Base ./Base.jl:420
  [5] include(x::String)
    @ PyCall ~/.julia/packages/PyCall/BD546/src/PyCall.jl:1
  [6] top-level scope
    @ ~/.julia/packages/PyCall/BD546/src/PyCall.jl:38
  [7] include
    @ ./Base.jl:420 [inlined]
  [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1318
  [9] top-level scope
    @ none:1
 [10] eval
    @ ./boot.jl:373 [inlined]
 [11] eval(x::Expr)
    @ Base.MainInclude ./client.jl:453
 [12] top-level scope
    @ none:1
ERROR: Failed to precompile PyCall [438e738f-606a-5dbb-bf0a-cddfbfd45ab0] to /Users/vsulzer/.julia/compiled/v1.7/PyCall/jl_3JtT1p.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
   @ Base ./loading.jl:1466
 [3] compilecache(pkg::Base.PkgId, path::String)
   @ Base ./loading.jl:1410
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1120
 [5] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1013
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:997

(@v1.7) pkg> build -v PyCall
    Building Conda ─→ `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/299304989a5e6473d985212c28928899c74e9421/build.log`
    Building PyCall → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/169bb8ea6b1b143c5cf57df6d34d022a7b60c6db/build.log`
[ Info: PyCall is using /usr/local/Caskroom/miniforge/base/envs/conda_jl/bin/python (Python 3.8.12) at /usr/local/Caskroom/miniforge/base/envs/conda_jl/bin/python, libpython = /System/Library/Frameworks/Python.framework/Versions/2.7/Python
[ Info: /Users/vsulzer/.julia/packages/PyCall/BD546/deps/deps.jl has been updated
[ Info: /Users/vsulzer/.julia/prefs/PyCall has not changed
Precompiling project...
  1 dependency successfully precompiled in 3 seconds (17 already precompiled)

julia> using PyCall
ImportError: No module named site
valentinsulzer commented 2 years ago

After building in 1.6, PyCall/deps/deps.jl is

const python = "/usr/local/Caskroom/miniforge/base/envs/conda_jl/bin/python"
const libpython = "/usr/local/Caskroom/miniforge/base/envs/conda_jl/lib/libpython3.8.dylib"
const pyprogramname = "/usr/local/Caskroom/miniforge/base/envs/conda_jl/bin/python"
const pyversion_build = v"3.8.12"
const PYTHONHOME = "/usr/local/Caskroom/miniforge/base/envs/conda_jl:/usr/local/Caskroom/miniforge/base/envs/conda_jl"

"True if we are using the Python distribution in the Conda package."
const conda = false

After building in 1.7:

const python = "/usr/local/Caskroom/miniforge/base/envs/conda_jl/bin/python"
const libpython = "/System/Library/Frameworks/Python.framework/Versions/2.7/Python"
const pyprogramname = "/usr/local/Caskroom/miniforge/base/envs/conda_jl/bin/python"
const pyversion_build = v"3.8.12"
const PYTHONHOME = "/usr/local/Caskroom/miniforge/base/envs/conda_jl:/usr/local/Caskroom/miniforge/base/envs/conda_jl"

"True if we are using the Python distribution in the Conda package."
const conda = false

So I guess the issue is the line

const libpython = "/System/Library/Frameworks/Python.framework/Versions/2.7/Python"
valentinsulzer commented 2 years ago

I've just seen that Mac M1 has Tier 3 support with Julia, so feel free to close this if it's not supported

LarryLegend33 commented 2 years ago

I have this exact problem. Encountered after homebrew upgrade to julia 1.7 from 1.6.

ImportError: No module named site

Process julia exited abnormally with code 1

cortner commented 2 years ago

same issue here. But I get it with both Julia 1.7 and 1.6 (where 1.6 runs via Rosetta)

Does anybody have a workaround yet?

cortner commented 2 years ago

FWIW - it seems that if I use the system python in /usr I don't have the problem. (but that python version has other issues that I can't resolve and I'd rather use anaconda or miniconda)

stevengj commented 2 years ago

libpython3.8.dylib: mach-o, but wrong architecture

You still have the Rosetta version of Python installed and it won't work with native M1 Julai. You need to re-install Anaconda with the native-M1 versions to use it from a native-M1 Julia.

Just rm -rf ~/.julia/conda and re-install.

cortner commented 2 years ago

I've tried this multiple times without success, sorry. Even deleting the entire .julia folder didn't help.

CaptainMalick commented 2 years ago

This issue doesn't really seem fixed, I have the same problem with Python installed for ARM

gaston-fernandez commented 9 months ago

I had a similar issue working with Julia (version 1.9) in Visual Code.

When using PyCall, Julia exited abnormally with code 1.

Re-installing Anaconda for the M1 version worked for me.