search

JuliaPy / Conda.jl

Conda managing Julia binary dependencies
Other
171 stars 57 forks source link

error on conda package imports #237

Closed deszoeke closed 1 year ago

deszoeke commented 1 year ago

I recently installed mamba (conda would not solve). Perhaps mamba interferes with .julia/conda, but I'm not sure. Using or pyimporting packages like numpy and matplotlib causes hangs or crashes.

Here's what I do:

bash> mamba env list
# conda environments:
#
                         ~/.julia/conda/3
base                  *  ~/mambaforge
ASTRAL_soundings         ~/mambaforge/envs/ASTRAL_soundings
Rayleigh                 ~/mambaforge/envs/Rayleigh

Recursively remove .julia/conda.

bash> env | grep CONDA
CONDA_SHLVL=1
CONDA_PROMPT_MODIFIER=(base) 
CONDA_EXE=$HOME/mambaforge/bin/conda
_CE_CONDA=
CONDA_PREFIX=$HOME/mambaforge
CONDA_PYTHON_EXE=$HOME/mambaforge/bin/python
CONDA_DEFAULT_ENV=base

Do these env variables conflict with .julia/conda? If so, how should I use mamba outside, and .juila/conda inside julia?

bash>  LD_LIBRARY_PATH=$HOME/.julia/conda/3/lib /Applications/Julia-1.9.app/Contents/Resources/julia/bin/julia

(julia v1.8.5 behaves the same.)

julia> ENV["PYTHON"]=""
(@v1.9) pkg> build Conda
    Building Conda → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/e32a90da027ca45d84678b826fffd3110bb3fc90/build.log`

Restart julia.

julia> ENV["PYTHON"]=""
(@v1.9) pkg> build PyCall
    Building Conda ─→ `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/e32a90da027ca45d84678b826fffd3110bb3fc90/build.log`
    Building PyCall → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/62f417f6ad727987c755549e9cd88c46578da562/build.log`

Restart julia.

julia> using Conda, PyCall; Conda.ROOTENV
"$HOME/.julia/conda/3"
julia > Conda.CONDA_EXE
"$HOME/.julia/conda/3/bin/conda"
julia> PyCall.PYTHONHOME
"$HOME/.julia/conda/3"

julia> pyimport("numpy")

hangs or dies with Killed: 9.

deszoeke commented 1 year ago

I resolved my problem with two steps.

  1. I commented out the conda setup that mamba adds to my .bash_profile. It exports many environment variables like CONDA_EXE, prepends ~/mambaforge/bin to the path and runs mamba.sh. Basically, I now run julia in a regular shell not from a mamba/conda environment (not even base).

This did not fix the problem with python dependencies. It might have helped to build the dependencies properly in step 2.

  1. I removed the subdirectories .julia/ 
    compiled
conda
artifacts
clones
packages
registries
scratchspaces

Then I was able to add and build PyCall, pyimport("numpy"), ]add PyPlot, and use it.