PyCall fails to build when using `PYTHON=""`

[rdeits]

On the latest Julia (1.8.5) with the latest release of PyCall (1.95.1), attempting to build PyCall with PYTHON="" (i.e. using the conda-provided python), fails:

julia> ENV["PYTHON"] = ""
""

(sympy-test) pkg> build PyCall
    Building Conda ─→ `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/e32a90da027ca45d84678b826fffd3110bb3fc90/build.log`
    Building PyCall → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/62f417f6ad727987c755549e9cd88c46578da562/build.log`
ERROR: Error building `PyCall`: 
Traceback (most recent call last):
  File "/home/rdeits/.julia/conda/3/bin/conda", line 7, in <module>
    from conda.cli import main
ModuleNotFoundError: No module named 'conda'
┌ Info: Using the Python distribution in the Conda package by default.
└ To use a different Python version, set ENV["PYTHON"]="pythoncommand" and re-run Pkg.build("PyCall").
[ Info: Running `conda install -y numpy` in root environment
ERROR: LoadError: failed process: Process(setenv(`/home/rdeits/.julia/conda/3/bin/conda install -y numpy`,
...

For completeness:

julia> versioninfo()
Julia Version 1.8.5
Commit 17cfb8e65ea (2023-01-08 06:45 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 8 × Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake)
  Threads: 1 on 8 virtual cores

(sympy-test) pkg> st
Status `~/Downloads/sympy-test/Project.toml`
  [438e738f] PyCall v1.95.1

This is on Ubuntu 20.04.

[deszoeke]

Mine does not error when compiling, but it fails when importing conda packages, in either julia 1.8.5 or julia 1.9.0.

LD_LIBRARY_PATH=$HOME/.julia/conda/3/lib /Applications/Julia-1.9.app/Contents/Resources/julia/bin/julia
julia> ENV["PYTHON"]=""
(@v1.9) pkg> build PyCall
    Building Conda ─→ `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/e32a90da027ca45d84678b826fffd3110bb3fc90/build.log`
    Building PyCall → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/62f417f6ad727987c755549e9cd88c46578da562/build.log`

restarting julia

julia> using PyCall
julia> py"""2+2"""
4
julia> py"""
import numpy
"""

hangs.

julia> py"""
import matplotlib
"""
Killed: 9

crashes.

julia> versioninfo()
Julia Version 1.9.0
Commit 8e630552924 (2023-05-07 11:25 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin22.4.0)
  CPU: 8 × Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, skylake)
  Threads: 1 on 8 virtual cores

(@v1.9) pkg> st
Status `~/.julia/environments/v1.9/Project.toml`
  [8f4d0f93] Conda v1.8.0
  [438e738f] PyCall v1.95.1
  [d330b81b] PyPlot v2.11.1

[stevengj]

Traceback (most recent call last):
  File "/home/rdeits/.julia/conda/3/bin/conda", line 7, in <module>
    from conda.cli import main
ModuleNotFoundError: No module named 'conda'

make it look like your conda installation is broken somehow? You could try removing /home/rdeits/.julia/conda and trying again?

It works for me…

[deszoeke]

I’m interested if this works. I did remove .julia/conda, but that didn’t fix it for me.

Anything else I can delete to remove any state or possible error for PyCall?

[stevengj]

In any case, it seems you are reporting the error in the wrong package — it seems like it is a problem with Conda.jl. e.g. does

using Conda
Conda.add("numpy")

work? What is the values of Conda.ROOTENV?

[rdeits]

@stevengj thanks--nuking ~/.julia/conda fixed the issue. I'm not sure how my system got into that state--manually rebuilding Conda wasn't enough without nuking that folder.

[deszoeke]

Thanks.

I had to add Conda. (Though I think it was already a dependency of PyCall.)

julia> Conda.add("numpy")
[ Info: Running `conda install -y numpy` in root environment
Retrieving notices: ...working... done
Collecting package metadata (current_repodata.json): done
Solving environment: done

# All requested packages already installed.

julia> Conda.ROOTENV
"$HOME/.julia/conda/3"

$HOME is my user home directory.

Rebuilt PyCall and restarted julia. pyimport("numpy") still hangs.