search

rafaqz / SegmentAnything.jl

segment-anything wrapped for Julia
MIT License
55 stars 2 forks source link

Failed to precompile SegmentAnything #4

Open chadagreene opened 1 year ago

chadagreene commented 1 year ago

Hya, I want to give SegmentAnything a go, but I'm running into an issue loading the package. Can you help me interpret this stacktrace or offer any guidance? 

julia> using SegmentAnything, GLMakie, FileIO
[ Info: Precompiling SegmentAnything [daca4e5c-8485-4416-9301-671ef5a159db]
    CondaPkg Found dependencies: /Users/cgreene/.julia/packages/PythonCall/qTEA1/CondaPkg.toml
    CondaPkg Found dependencies: /Users/cgreene/.julia/packages/SegmentAnything/uDUsK/CondaPkg.toml
    CondaPkg Resolving changes
             + numpy
             + opencv
             + pip
             + python
             + pytorch
             + pytorch-cuda
             + segment-anything (pip)
             + torchaudio
             + torchvision
    CondaPkg Creating environment
             │ /Users/cgreene/.julia/artifacts/6ecf04294c7f327e02e84972f34835649a5eb35e/bin/micromamba
             │ -r /Users/cgreene/.julia/scratchspaces/0b3b1443-0f03-428d-bdfb-f27f9c1191ea/root
             │ create
             │ -y
             │ -p /Users/cgreene/.julia/environments/v1.9/.CondaPkg/env
             │ --override-channels
             │ --no-channel-priority
             │ numpy[version='*']
             │ opencv[version='*']
             │ pip[version='>=22.0.0']
             │ python[version='>=3.7,<4',channel='conda-forge',build='*cpython*']
             │ pytorch[version='*']
             │ pytorch-cuda[version='*']
             │ torchaudio[version='*']
             │ torchvision[version='*']
             │ -c conda-forge
             │ -c nvidia
             └ -c pytorch
conda-forge/osx-64                                          Using cache
conda-forge/noarch                                          Using cache
nvidia/osx-64                                               Using cache
nvidia/noarch                                               Using cache
pytorch/osx-64                                              Using cache
pytorch/noarch                                              Using cache
error    libmamba Could not solve for environment specs
    The following packages are incompatible
    └─ pytorch-cuda *  is not installable because there are no viable options
       ├─ pytorch-cuda 11.6 would require
       │  └─ cuda 11.6.* , which does not exist (perhaps a missing channel);
       ├─ pytorch-cuda 11.7 would require
       │  └─ cuda 11.7.* , which does not exist (perhaps a missing channel);
       └─ pytorch-cuda 11.8 would require
          └─ cuda 11.8.* , which does not exist (perhaps a missing channel).
critical libmamba Could not solve for environment specs
ERROR: LoadError: InitError: failed process: Process(`/Users/cgreene/.julia/artifacts/6ecf04294c7f327e02e84972f34835649a5eb35e/bin/micromamba -r /Users/cgreene/.julia/scratchspaces/0b3b1443-0f03-428d-bdfb-f27f9c1191ea/root create -y -p /Users/cgreene/.julia/environments/v1.9/.CondaPkg/env --override-channels --no-channel-priority "numpy[version='*']" "opencv[version='*']" "pip[version='>=22.0.0']" "python[version='>=3.7,<4',channel='conda-forge',build='*cpython*']" "pytorch[version='*']" "pytorch-cuda[version='*']" "torchaudio[version='*']" "torchvision[version='*']" -c conda-forge -c nvidia -c pytorch`, ProcessExited(1)) [1]

Stacktrace:
  [1] pipeline_error
    @ ./process.jl:565 [inlined]
  [2] run(::Cmd; wait::Bool)
    @ Base ./process.jl:480
  [3] run(::Cmd)
    @ Base ./process.jl:477
  [4] _run(io::IO, cmd::Cmd, args::Any; flags::Any)
    @ CondaPkg ~/.julia/packages/CondaPkg/osUdN/src/resolve.jl:370
  [5] _resolve_conda_install(io::Any, conda_env::Any, specs::Any, channels::Any; create::Any)
    @ CondaPkg ~/.julia/packages/CondaPkg/osUdN/src/resolve.jl:271
  [6] resolve(; force::Bool, io::IO, interactive::Bool, dry_run::Bool)
    @ CondaPkg ~/.julia/packages/CondaPkg/osUdN/src/resolve.jl:526
  [7] resolve()
    @ CondaPkg ~/.julia/packages/CondaPkg/osUdN/src/resolve.jl:384
  [8] envdir()
    @ CondaPkg ~/.julia/packages/CondaPkg/osUdN/src/env.jl:70
  [9] init_context()
    @ PythonCall.C ~/.julia/packages/PythonCall/qTEA1/src/cpython/context.jl:63
 [10] __init__()
    @ PythonCall.C ~/.julia/packages/PythonCall/qTEA1/src/cpython/CPython.jl:21
 [11] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1115
 [12] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any})
    @ Base ./loading.jl:1061
 [13] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
    @ Base ./loading.jl:1506
 [14] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1783
 [15] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1660
 [16] macro expansion
    @ ./loading.jl:1648 [inlined]
 [17] macro expansion
    @ ./lock.jl:267 [inlined]
 [18] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1611
 [19] include
    @ ./Base.jl:457 [inlined]
 [20] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2049
 [21] top-level scope
    @ stdin:3
during initialization of module C
in expression starting at /Users/cgreene/.julia/packages/SegmentAnything/uDUsK/src/SegmentAnything.jl:1
in expression starting at stdin:3
ERROR: Failed to precompile SegmentAnything [daca4e5c-8485-4416-9301-671ef5a159db] to "/Users/cgreene/.julia/compiled/v1.9/SegmentAnything/jl_veZKKp".
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:2300
  [3] compilecache
    @ ./loading.jl:2167 [inlined]
  [4] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1805
  [5] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1660
  [6] macro expansion
    @ ./loading.jl:1648 [inlined]
  [7] macro expansion
    @ ./lock.jl:267 [inlined]
  [8] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1611
  [9] eval
    @ ./boot.jl:370 [inlined]
 [10] eval
    @ ./Base.jl:68 [inlined]
 [11] repleval(m::Module, code::Expr, #unused#::String)
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.47.2/scripts/packages/VSCodeServer/src/repl.jl:222
 [12] (::VSCodeServer.var"#107#109"{Module, Expr, REPL.LineEditREPL, REPL.LineEdit.Prompt})()
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.47.2/scripts/packages/VSCodeServer/src/repl.jl:186
 [13] with_logstate(f::Function, logstate::Any)
    @ Base.CoreLogging ./logging.jl:514
 [14] with_logger
    @ ./logging.jl:626 [inlined]
 [15] (::VSCodeServer.var"#106#108"{Module, Expr, REPL.LineEditREPL, REPL.LineEdit.Prompt})()
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.47.2/scripts/packages/VSCodeServer/src/repl.jl:187
 [16] #invokelatest#2
    @ ./essentials.jl:816 [inlined]
 [17] invokelatest(::Any)
    @ Base ./essentials.jl:813
 [18] macro expansion
    @ ~/.vscode/extensions/julialang.language-julia-1.47.2/scripts/packages/VSCodeServer/src/eval.jl:34 [inlined]
 [19] (::VSCodeServer.var"#61#62")()
    @ VSCodeServer ./task.jl:514

julia> predictor = SamPredictor()
ERROR: UndefVarError: `SamPredictor` not defined
Stacktrace:
 [1] top-level scope
   @ REPL[2]:1

julia>
rafaqz commented 1 year ago

This is all on the python side... you can see the error is libmamba not resolving the dependencies. Im not sure what they have changed, their versioning is hell.

But you could clone the package and try to update the python package versions?

Probably faster than waiting for me to get time for it.

(Seems like there is a problem with the channel for cuda?)

chadagreene commented 1 year ago

Awesome, I'll give that a shot. Thanks @rafaqz!

rafaqz commented 1 year ago

Did you get this working?

chadagreene commented 1 year ago

@rafaqz Thanks for asking. I'm afraid I haven't gotten it working yet. I spent a morning trying to resolve conflicts on the Python side but couldn't quite get over this hump:

Package anyio conflicts for:
anaconda==2021.11 -> jupyter_server==1.4.1=py37hecd8cb5_0 -> anyio[version='>=2.0.2']
jupyterlab -> jupyter_server[version='>=1.16,<3'] -> anyio[version='>=2.0.2|>=3.1.0,<4']
jupyterlab_server -> jupyter_server[version='>=1.21,<3'] -> anyio[version='>=2.0.2|>=3.1.0,<4']
anaconda==2021.11 -> anyio==2.2.0[build='py38hecd8cb5_1|py37hecd8cb5_1|py39hecd8cb5_1']
nbclassic -> jupyter_server[version='>=1.8'] -> anyio[version='>=2.0.2|>=3.1.0,<4']
jupyter_server -> anyio[version='>=2.0.2|>=3.1.0,<4']

Package nspr conflicts for:
pyqt -> qt-main=5.15 -> nspr[version='>=4.33,<5.0a0']
qt -> qt-main[version='>=5.15,<6'] -> nspr[version='>=4.33,<5.0a0']

Package selectors2 conflicts for:
wurlitzer -> selectors2
spyder-kernels -> wurlitzer[version='>=1.0.3'] -> selectors2

Package watchdog conflicts for:
spyder -> watchdog[version='>=0.10.3|>=0.10.3,<2.0.0']
anaconda==2021.11 -> spyder==5.1.5=py37hecd8cb5_1 -> watchdog[version='>=0.10.3']
anaconda==2021.11 -> watchdog==2.1.3[build='py37hd5f6b7e_0|py38hd5f6b7e_0|py39hd5f6b7e_0']

Package pkgutil-resolve-name conflicts for:
spyder -> jsonschema[version='>=3.2.0'] -> pkgutil-resolve-name[version='>=1.3.10']
jupyterlab_server -> jsonschema[version='>=4.17.3'] -> pkgutil-resolve-name[version='>=1.3.10']
nbformat -> jsonschema[version='>=2.6'] -> pkgutil-resolve-name[version='>=1.3.10']
jsonschema -> pkgutil-resolve-name[version='>=1.3.10']

Package jupyter_console conflicts for:
anaconda==2021.11 -> jupyter==1.0.0=py37_7 -> jupyter_console
anaconda==2021.11 -> jupyter_console==6.4.0=pyhd3eb1b0_0
jupyter -> jupyter_console

Package geos conflicts for:
shapely -> geos[version='>=3.10.1,<3.10.2.0a0|>=3.8.0,<3.8.1.0a0|>=3.7.1,<3.7.2.0a0|>=3.6.2,<3.6.3.0a0|>=3.6.0,<3.6.1.0a0']
ipyleaflet -> shapely -> geos[version='>=3.10.1,<3.10.2.0a0|>=3.8.0,<3.8.1.0a0|>=3.7.1,<3.7.2.0a0|>=3.6.2,<3.6.3.0a0|>=3.6.0,<3.6.1.0a0']

Package keyring conflicts for:
spyder -> keyring[version='>=17.0.0']
anaconda==2021.11 -> keyring==23.1.0[build='py39hecd8cb5_0|py38hecd8cb5_0|py37hecd8cb5_0']
anaconda==2021.11 -> spyder==5.1.5=py37hecd8cb5_1 -> keyring[version='>=17.0.0']

Package conda-pack conflicts for:
anaconda-project -> conda-pack
anaconda==2021.11 -> anaconda-project==0.10.1=pyhd3eb1b0_0 -> conda-pack
anaconda-navigator -> anaconda-project -> conda-pack

Package dask conflicts for:
anaconda==2021.11 -> dask==2021.10.0=pyhd3eb1b0_0
scikit-image -> dask[version='>=0.5']

Package glob2 conflicts for:
conda-build -> glob2[version='>=0.6']
anaconda==2021.11 -> glob2==0.7=pyhd3eb1b0_0

Package libllvm11 conflicts for:
anaconda==2021.11 -> libllvm11==11.1.0=h9b2ccf5_0
llvmlite -> libllvm11[version='>=11.0.0,<11.1.0a0|>=11.1.0,<11.2.0a0']
anaconda==2021.11 -> llvmlite==0.37.0=py37he4411ff_1 -> libllvm11[version='>=11.1.0,<11.2.0a0']
numba -> llvmlite[version='>=0.39.*,<0.40'] -> libllvm11[version='>=11.0.0,<11.1.0a0|>=11.1.0,<11.2.0a0']

Package ripgrep conflicts for:
conda-build -> ripgrep
anaconda==2021.11 -> ripgrep==12.1.1=0

Package unidecode conflicts for:
anaconda==2021.11 -> unidecode==1.2.0=pyhd3eb1b0_0
cookiecutter -> python-slugify[version='>=4.0.0'] -> unidecode[version='>=1.1.1']
python-slugify -> unidecode[version='>=0.04.16|>=1.1.1']
anaconda==2021.11 -> python-slugify==5.0.2=pyhd3eb1b0_0 -> unidecode[version='>=1.1.1']

Package curl conflicts for:
pycurl -> curl[version='7.55.*|>=7.55.1,<9.0a0']
anaconda==2021.11 -> curl==7.78.0=h7bc2e8c_0

Package pyrsistent conflicts for:
jsonschema -> pyrsistent[version='!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0|>=0.14.0,!=0.17.0,!=0.17.1,!=0.17.2|>=0.14.0']
anaconda==2021.11 -> pyrsistent==0.18.0[build='py39hca72f7f_0|py37hca72f7f_0|py38hca72f7f_0']
nbformat -> jsonschema[version='>=2.6'] -> pyrsistent[version='!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0|>=0.14.0,!=0.17.0,!=0.17.1,!=0.17.2|>=0.14.0']
spyder -> jsonschema[version='>=3.2.0'] -> pyrsistent[version='!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0|>=0.14.0,!=0.17.0,!=0.17.1,!=0.17.2|>=0.14.0']
jupyterlab_server -> jsonschema[version='>=4.17.3'] -> pyrsistent[version='!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0|>=0.14.0,!=0.17.0,!=0.17.1,!=0.17.2|>=0.14.0']
anaconda==2021.11 -> jsonschema==3.2.0=pyhd3eb1b0_2 -> pyrsistent[version='>=0.14.0']

Package html5lib conflicts for:
anaconda==2021.11 -> html5lib==1.1=pyhd3eb1b0_0
nbconvert -> bleach -> html5lib[version='>=0.99999999|>=0.99999999pre,!=1.0b1,!=1.0b2,!=1.0b3,!=1.0b4,!=1.0b5,!=1.0b6,!=1.0b7,!=1.0b8']
bleach -> html5lib[version='>=0.99999999|>=0.99999999pre,!=1.0b1,!=1.0b2,!=1.0b3,!=1.0b4,!=1.0b5,!=1.0b6,!=1.0b7,!=1.0b8']

Package unixodbc conflicts for:
pyodbc -> unixodbc[version='2.3.4.*|>=2.3.11,<2.4.0a0|>=2.3.9,<2.4.0a0|>=2.3.7,<2.4.0a0|>=2.3.6,<2.4.0a0']
anaconda==2021.11 -> pyodbc==4.0.31=py37h23ab428_0 -> unixodbc[version='>=2.3.9,<2.4.0a0']
anaconda==2021.11 -> unixodbc==2.3.9=haf1e3a3_0

Package statsmodels conflicts for:
seaborn -> statsmodels[version='>=0.5.0']
anaconda==2021.11 -> statsmodels==0.12.2[build='py39h9ed2024_0|py37h9ed2024_0|py38h9ed2024_0']

Package textdistance conflicts for:
anaconda==2021.11 -> spyder==5.1.5=py37hecd8cb5_1 -> textdistance[version='>=4.2.0']
anaconda==2021.11 -> textdistance==4.2.1=pyhd3eb1b0_0
spyder -> textdistance[version='>=4.2.0']

Package ipywidgets conflicts for:
anaconda==2021.11 -> ipywidgets==7.6.5=pyhd3eb1b0_1
ipyleaflet -> ipywidgets[version='>=7.6.0,<8.0.0']
jupyter -> ipywidgets
anaconda==2021.11 -> jupyter==1.0.0=py37_7 -> ipywidgets
_ipyw_jlab_nb_ext_conf -> ipywidgets

Package libllvm9 conflicts for:
llvmlite -> libllvm9==9.0.1[build=*_1]
numba -> llvmlite[version='>=0.33.0',build=*_1] -> libllvm9==9.0.1[build=*_1]

Package rtree conflicts for:
anaconda==2021.11 -> rtree==0.9.7[build='py38hecd8cb5_1|py37hecd8cb5_1|py39hecd8cb5_1']
anaconda==2021.11 -> spyder==5.1.5=py37hecd8cb5_1 -> rtree[version='>=0.9.7']
spyder -> rtree[version='>=0.8.3|>=0.9.7']

Package pytest conflicts for:
anaconda==2021.11 -> pytest==6.2.4[build='py37hecd8cb5_2|py38hecd8cb5_2|py39hecd8cb5_2']
astropy -> pytest[version='<3.7|<4']
astropy -> pytest-astropy -> pytest[version='>=3.1.0|>=3.1|>=4.6']

Package cached-property conflicts for:
anaconda==2021.11 -> cached-property==1.5.2=py_0
anaconda==2021.11 -> h5py==3.2.1=py39h90fc2a2_0 -> cached-propertyThe following specifications were found to be incompatible with your system:

  - feature:/osx-64::__osx==10.16=0
  - feature:|@/osx-64::__osx==10.16=0
  - matplotlib -> matplotlib-base[version='>=3.7.1,<3.7.2.0a0'] -> __osx[version='>=10.11']
  - matplotlib-base -> __osx[version='>=10.11']
  - scikit-image -> matplotlib-base[version='>=2.0.0,!=3.0.0'] -> __osx[version='>=10.11']
  - seaborn -> matplotlib-base[version='>=3.1,!=3.6.1'] -> __osx[version='>=10.11']
rafaqz commented 1 year ago

Hmm those are a weird mix of deps, I guess theyre shared accross all of the packages using PythonCall.jl in your session? Or are you using system Conda?

I had hoped these things were isolated from each other somehow but I guess that was wishful thinking.

terasakisatoshi commented 6 months ago

I'm working on my machine (Intel macOS). The following CondaPkg.toml should solve our issue.

channels = ["conda-forge", "pytorch"]

[deps]
opencv = ""
pytorch = ""
torchvision = ""
numpy = ""
# https://discuss.pytorch.org/t/intel-mkl-fatal-error-cannot-load-libmkl-core-dylib/67710
nomkl = ""
torchaudio = ""

[pip.deps]
segment-anything = "@git+https://github.com/facebookresearch/segment-anything.git"