issue with cellpose - Githubissues

haesleinhuepf commented 2 years ago

It appears, cellpose is currently not correctly installing together with devbio-napari:

On @kevinyamauchi's computer there is an error, see details https://github.com/conda-forge/devbio-napari-feedstock/pull/17#issuecomment-1250040154

On mine the installation works:

mamba create --name devbio-napari-env python=3.9 devbio-napari -c conda-forge

And then I receive an error when running cellpose from the tools menu on blobs.gif:

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
File ~\miniconda3\envs\devbio-napari-env4\lib\site-packages\napari_tools_menu\__init__.py:71, in ToolsMenu.make_sub_sub_menu.<locals>.func(whatever=False)
     66             break
     67         # cannot look for param.kind == param.VAR_KEYWORD because
     68         # QWidget allows **kwargs but errs on unknown keyword arguments
     69
     70     # instantiate the widget
---> 71     wdg = action(**kwargs)
        action = <function widget_wrapper at 0x000002004BBB48B0>
        kwargs = {}
     72     dw = napari_viewer.window.add_dock_widget(wdg, name=title)
     73 if dw is not None:
     74     # workaround for https://github.com/napari/napari/issues/4348

File ~\miniconda3\envs\devbio-napari-env4\lib\site-packages\cellpose_napari\_dock_widget.py:50, in widget_wrapper()
     48 from napari.qt.threading import thread_worker
     49 try:
---> 50     from torch import no_grad
     51 except ImportError:
     52     def no_grad():

File ~\miniconda3\envs\devbio-napari-env4\lib\site-packages\torch\__init__.py:124
    122     err = ctypes.WinError(last_error)
    123     err.strerror += f' Error loading "{dll}" or one of its dependencies.'
--> 124     raise err
        err = OSError(22, 'The operating system cannot run %1.', None, 182)
    125 elif res is not None:
    126     is_loaded = True

OSError: [WinError 182] The operating system cannot run %1. Error loading "C:\Users\rober\miniconda3\envs\devbio-napari-env4\lib\site-packages\torch\lib\caffe2_detectron_ops.dll" or one of its dependencies.

haesleinhuepf commented 2 years ago

I propose removing cellpose from devbio-napari if there is no easy solution to this.

kevinyamauchi commented 2 years ago

I propose removing cellpose from devbio-napari if there is no easy solution to this.

That would work for me for now.

That being said, I would like to try and figure out the best way to allow a pytorch dependency as that is going to be increasingly "required" for quality segmentations and other tasks. pytorch has been my preferred deep learning library for some time, but with the move to transfer the library to the linux foundation, I am even more inclined to depend on pytorch over tensorflow.

In general, I think it could make sense to define version constraints versions for key dependencies that are likely to cause conflicts in the devbio-napari ecosystem and explicitly install them. Then plugin developers who wish to be included in the devbio-napari "distribution" know what they need to follow.

Some potential candidates:

torch
numba
numpy

haesleinhuepf commented 2 years ago

In general, I think it could make sense to define version constraints versions for key dependencies that are likely to cause conflicts in the devbio-napari ecosystem and explicitly install them.

Yes! That goes very well in line with our local discussions. CC @thawn @zoccoler @jo-mueller

kevinyamauchi commented 2 years ago

Yes! That goes very well in line with our local discussions. CC @thawn @zoccoler @jo-mueller

Cool, glad we're thinking along the same lines!

To get the ball rolling, I propose the following (I haven't tested if this conflicts with any existing dependencies):

numpy == 1.22.4 (most recent version compatible with numba 0.55.2)
numba == 0.55.2 (most recent version available on conda-forge)
~~torch == 1.8.2 (this is the long term service version - seems reasonable?)~~ actually it looks like the LTS is compatible with MacOS and requires python <= 3.8
torch == 1.12.1 (this is the current "stable" version)

It could be worth keeping an eye on the BioImage Model Zoo, as they might specify pytorch versions and it would be nice to be in line with that project, as I imagine there will be a bunch of useful models available there in the coming months/years.

kevinyamauchi commented 2 years ago

Actually, I just ran into this issue when trying to install pytorch 1.12.1 on my mac. Maybe it's best to use 1.11.0 or maybe even older.

https://github.com/pytorch/pytorch/issues/78490#issuecomment-1217134036

haesleinhuepf commented 2 years ago

I've just tried to make devbio-napari work together with cellpose and failed. I've tried installation commands similar to those on a pretty emptyish windows computer:

mamba create --name devbio-napari-env python=3.9 devbio-napari pytorch redlionfish mkl_fft cpuonly cellpose-napari -c conda-forge -c pytorch

When starting cellpose from within napari, I receive this error:

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
File C:\Programs\miniconda3\envs\devbio-napari-env2\lib\site-packages\napari_tools_menu\__init__.py:71, in ToolsMenu.make_sub_sub_menu.<locals>.func(whatever=False)
     66             break
     67         # cannot look for param.kind == param.VAR_KEYWORD because
     68         # QWidget allows **kwargs but errs on unknown keyword arguments
     69
     70     # instantiate the widget
---> 71     wdg = action(**kwargs)
        action = <function widget_wrapper at 0x000001E0EC04E700>
        kwargs = {}
     72     dw = napari_viewer.window.add_dock_widget(wdg, name=title)
     73 if dw is not None:
     74     # workaround for https://github.com/napari/napari/issues/4348

File C:\Programs\miniconda3\envs\devbio-napari-env2\lib\site-packages\cellpose_napari\_dock_widget.py:50, in widget_wrapper()
     48 from napari.qt.threading import thread_worker
     49 try:
---> 50     from torch import no_grad
     51 except ImportError:
     52     def no_grad():

File C:\Programs\miniconda3\envs\devbio-napari-env2\lib\site-packages\torch\__init__.py:129
    127     err = ctypes.WinError(last_error)
    128     err.strerror += f' Error loading "{dll}" or one of its dependencies.'
--> 129     raise err
        err = OSError(22, 'The operating system cannot run %1.', None, 182)
    130 elif res is not None:
    131     is_loaded = True

OSError: [WinError 182] The operating system cannot run %1. Error loading "C:\Programs\miniconda3\envs\devbio-napari-env2\lib\site-packages\torch\lib\shm.dll" or one of its dependencies.

This is the conda list:

# packages in environment at C:\Programs\miniconda3\envs\devbio-napari-env2:
#
# Name                    Version                   Build  Channel
alabaster                 0.7.12                     py_0    conda-forge
anyio                     3.6.1              pyhd8ed1ab_1    conda-forge
aom                       3.5.0                h63175ca_0    conda-forge
apoc-backend              0.10.0             pyhd8ed1ab_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
argon2-cffi               21.3.0             pyhd8ed1ab_0    conda-forge
argon2-cffi-bindings      21.2.0           py39hb82d6ee_2    conda-forge
asciitree                 0.3.3                      py_2    conda-forge
asttokens                 2.0.8              pyhd8ed1ab_0    conda-forge
attrs                     22.1.0             pyh71513ae_1    conda-forge
autopep8                  1.7.0              pyhd8ed1ab_0    conda-forge
babel                     2.10.3             pyhd8ed1ab_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                        py_2    conda-forge
backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
beautifulsoup4            4.11.1             pyha770c72_0    conda-forge
blas                      2.116                       mkl    conda-forge
blas-devel                3.9.0              16_win64_mkl    conda-forge
bleach                    5.0.1              pyhd8ed1ab_0    conda-forge
blosc                     1.21.1               h74325e0_3    conda-forge
bokeh                     2.4.3              pyhd8ed1ab_3    conda-forge
brotli                    1.0.9                h8ffe710_7    conda-forge
brotli-bin                1.0.9                h8ffe710_7    conda-forge
brotlipy                  0.7.0           py39hb82d6ee_1004    conda-forge
build                     0.7.0              pyhd8ed1ab_0    conda-forge
bzip2                     1.0.8                h8ffe710_4    conda-forge
c-blosc2                  2.4.2                h183a6f4_0    conda-forge
ca-certificates           2022.9.24            h5b45459_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachey                    0.2.1              pyh9f0ad1d_0    conda-forge
cellpose                  2.0.5              pyhd8ed1ab_0    conda-forge
cellpose-napari           0.1.5              pyhd8ed1ab_0    conda-forge
certifi                   2022.9.24          pyhd8ed1ab_0    conda-forge
cffi                      1.15.1           py39h0878f49_0    conda-forge
cfitsio                   4.1.0                h5a969a9_0    conda-forge
charls                    2.3.4                h39d44d4_0    conda-forge
charset-normalizer        2.1.1              pyhd8ed1ab_0    conda-forge
click                     8.1.3            py39hcbf5309_0    conda-forge
click-default-group       1.2.2              pyhd8ed1ab_1    conda-forge
cloudpickle               2.2.0              pyhd8ed1ab_0    conda-forge
colorama                  0.4.5              pyhd8ed1ab_0    conda-forge
commonmark                0.9.1                      py_0    conda-forge
contourpy                 1.0.5            py39h1f6ef14_0    conda-forge
control                   0.9.2              pyhd8ed1ab_0    conda-forge
cpuonly                   2.0                           0    pytorch
cryptography              37.0.4           py39h7bc7c5c_0    conda-forge
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
cytoolz                   0.12.0           py39hb82d6ee_0    conda-forge
czifile                   2019.7.2           pyh9f0ad1d_0    conda-forge
dask                      2022.9.2           pyhd8ed1ab_0    conda-forge
dask-core                 2022.9.2           pyhd8ed1ab_0    conda-forge
dataclasses               0.8                pyhc8e2a94_3    conda-forge
dav1d                     1.0.0                h8ffe710_1    conda-forge
debugpy                   1.6.3            py39h415ef7b_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
deprecated                1.2.13             pyh6c4a22f_0    conda-forge
devbio-napari             0.7.0            win_h08f2357_0    conda-forge
distributed               2022.9.2           pyhd8ed1ab_0    conda-forge
docstring_parser          0.13               pyhd8ed1ab_0    conda-forge
docutils                  0.19             py39hcbf5309_0    conda-forge
eigen                     3.4.0                h2d74725_0    conda-forge
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
executing                 1.1.0              pyhd8ed1ab_0    conda-forge
expat                     2.4.9                h1537add_0    conda-forge
fasteners                 0.17.3             pyhd8ed1ab_0    conda-forge
fastremap                 1.13.3           py39hefe7e4c_0    conda-forge
ffmpeg                    5.1.2           gpl_h952562b_101    conda-forge
fftw                      3.3.10          nompi_h38027f0_105    conda-forge
finesse                   3.0a3            py39h1cfdcae_2    conda-forge
flit-core                 3.7.1              pyhd8ed1ab_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.0               h720f74d_1    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.37.4           py39ha55989b_0    conda-forge
freeglut                  3.2.2                h0e60522_1    conda-forge
freetype                  2.12.1               h546665d_0    conda-forge
freetype-py               2.3.0              pyhd8ed1ab_0    conda-forge
fsspec                    2022.8.2           pyhd8ed1ab_0    conda-forge
funcsigs                  1.0.2                      py_3    conda-forge
future                    0.18.2           py39hcbf5309_5    conda-forge
gettext                   0.19.8.1          h5728263_1009    conda-forge
giflib                    5.2.1                h8d14728_2    conda-forge
glib                      2.74.0               h12be248_0    conda-forge
glib-tools                2.74.0               h12be248_0    conda-forge
gst-plugins-base          1.20.3               h001b923_2    conda-forge
gstreamer                 1.20.3               h6b5321d_2    conda-forge
h5py                      3.7.0           nompi_py39h7fe8f52_101    conda-forge
hdbscan                   0.8.28           py39h5d4886f_1    conda-forge
hdf5                      1.12.2          nompi_h2a0e4a3_100    conda-forge
heapdict                  1.0.1                      py_0    conda-forge
hsluv                     5.0.2              pyh44b312d_0    conda-forge
icu                       70.1                 h0e60522_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
imagecodecs               2022.9.26        py39h28bd840_0    conda-forge
imagecodecs-lite          2019.12.3        py39h5d4886f_5    conda-forge
imageio                   2.22.0             pyhfa7a67d_0    conda-forge
imageio-ffmpeg            0.4.7              pyhd8ed1ab_0    conda-forge
imagesize                 1.4.1              pyhd8ed1ab_0    conda-forge
importlib-metadata        4.11.4           py39hcbf5309_0    conda-forge
importlib_resources       5.9.0              pyhd8ed1ab_0    conda-forge
iniconfig                 1.1.1              pyh9f0ad1d_0    conda-forge
intel-openmp              2022.1.0          h57928b3_3787    conda-forge
ipycanvas                 0.13.1             pyhd8ed1ab_0    conda-forge
ipyevents                 2.0.1              pyhd8ed1ab_0    conda-forge
ipykernel                 6.16.0             pyh025b116_0    conda-forge
ipython                   8.5.0              pyh08f2357_1    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
ipywidgets                8.0.2              pyhd8ed1ab_1    conda-forge
jasper                    2.0.33               h77af90b_0    conda-forge
jedi                      0.18.1             pyhd8ed1ab_2    conda-forge
jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
joblib                    1.2.0              pyhd8ed1ab_0    conda-forge
jpeg                      9e                   h8ffe710_2    conda-forge
json5                     0.9.5              pyh9f0ad1d_0    conda-forge
jsonschema                4.16.0             pyhd8ed1ab_0    conda-forge
jupyter                   1.0.0            py39hcbf5309_7    conda-forge
jupyter_client            7.3.4              pyhd8ed1ab_0    conda-forge
jupyter_console           6.4.4              pyhd8ed1ab_0    conda-forge
jupyter_core              4.11.1           py39hcbf5309_0    conda-forge
jupyter_server            1.19.1             pyhd8ed1ab_0    conda-forge
jupyterlab                3.4.7              pyhd8ed1ab_0    conda-forge
jupyterlab_pygments       0.2.2              pyhd8ed1ab_0    conda-forge
jupyterlab_server         2.15.2             pyhd8ed1ab_0    conda-forge
jupyterlab_widgets        3.0.3              pyhd8ed1ab_0    conda-forge
jupytext                  1.14.0             pyheef035f_0    conda-forge
jxrlib                    1.1                  h8ffe710_2    conda-forge
khronos-opencl-icd-loader 2022.09.30           h64bf75a_0    conda-forge
kiwisolver                1.4.4            py39h2e07f2f_0    conda-forge
krb5                      1.19.3               h1176d77_0    conda-forge
lcms2                     2.12                 h2a16943_0    conda-forge
lerc                      4.0.0                h63175ca_0    conda-forge
libaec                    1.0.6                h39d44d4_0    conda-forge
libavif                   0.10.1               h8f5c5bc_2    conda-forge
libblas                   3.9.0              16_win64_mkl    conda-forge
libbrotlicommon           1.0.9                h8ffe710_7    conda-forge
libbrotlidec              1.0.9                h8ffe710_7    conda-forge
libbrotlienc              1.0.9                h8ffe710_7    conda-forge
libcblas                  3.9.0              16_win64_mkl    conda-forge
libclang                  14.0.6          default_h77d9078_0    conda-forge
libclang13                14.0.6          default_h77d9078_0    conda-forge
libcurl                   7.83.1               h789b8ee_0    conda-forge
libdeflate                1.14                 hcfcfb64_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libflang                  11.0.1          h0e60522_20210131    conda-forge
libglib                   2.74.0               h79619a9_0    conda-forge
libiconv                  1.17                 h8ffe710_0    conda-forge
libitk                    5.2.0                hac70a04_4    conda-forge
libitk-devel              5.2.0                h3bb2308_4    conda-forge
liblapack                 3.9.0              16_win64_mkl    conda-forge
liblapacke                3.9.0              16_win64_mkl    conda-forge
libogg                    1.3.4                h8ffe710_1    conda-forge
libopencv                 4.6.0            py39he11e404_0    conda-forge
libpng                    1.6.38               h19919ed_0    conda-forge
libprotobuf               3.20.1               h12be248_4    conda-forge
libsodium                 1.0.18               h8d14728_1    conda-forge
libsqlite                 3.39.3               hcfcfb64_0    conda-forge
libssh2                   1.10.0               h680486a_3    conda-forge
libtiff                   4.4.0                h8e97e67_4    conda-forge
libuv                     1.44.2               h8ffe710_0    conda-forge
libvorbis                 1.3.7                h0e60522_0    conda-forge
libwebp-base              1.2.4                h8ffe710_0    conda-forge
libxcb                    1.13              hcd874cb_1004    conda-forge
libxml2                   2.9.14               hf5bbc77_4    conda-forge
libxslt                   1.1.35               h34f844d_0    conda-forge
libzlib                   1.2.12               hcfcfb64_3    conda-forge
libzopfli                 1.0.3                h0e60522_0    conda-forge
llvm-openmp               11.0.1               h2d74725_0    conda-forge
llvmlite                  0.39.1           py39hd28a505_0    conda-forge
locket                    1.0.0              pyhd8ed1ab_0    conda-forge
loguru                    0.6.0            py39hcbf5309_1    conda-forge
lxml                      4.9.1            py39hb82d6ee_0    conda-forge
lz4                       4.0.0            py39h0878066_2    conda-forge
lz4-c                     1.9.3                h8ffe710_1    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
magicgui                  0.5.1              pyhd8ed1ab_0    conda-forge
mako                      1.2.3              pyhd8ed1ab_0    conda-forge
markdown-it-py            2.1.0              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.1            py39hb82d6ee_1    conda-forge
matplotlib-base           3.6.0            py39haf65ace_0    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
mdit-py-plugins           0.3.1              pyhd8ed1ab_0    conda-forge
mdurl                     0.1.0              pyhd8ed1ab_0    conda-forge
mistune                   2.0.4              pyhd8ed1ab_0    conda-forge
mkl                       2022.1.0           h6a75c08_874    conda-forge
mkl-devel                 2022.1.0           h57928b3_875    conda-forge
mkl-include               2022.1.0           h6a75c08_874    conda-forge
mkl_fft                   1.3.1            py39hb3d4d92_3    conda-forge
more-itertools            8.14.0             pyhd8ed1ab_0    conda-forge
mpmath                    1.2.1              pyhd8ed1ab_0    conda-forge
msgpack-python            1.0.4            py39h2e07f2f_0    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
napari                    0.4.15          pyh275ddea_1_pyqt    conda-forge
napari-3d-ortho-viewer    0.0.1              pyhd8ed1ab_0    conda-forge
napari-accelerated-pixel-and-object-classification 0.10.1             pyhd8ed1ab_0    conda-forge
napari-animation          0.0.2              pyhd8ed1ab_0    conda-forge
napari-assistant          0.3.12             pyhd8ed1ab_0    conda-forge
napari-blob-detection     0.0.2              pyhd8ed1ab_0    conda-forge
napari-brightness-contrast 0.1.8              pyhd8ed1ab_0    conda-forge
napari-clusters-plotter   0.5.2              pyhd8ed1ab_0    conda-forge
napari-console            0.0.6              pyhd8ed1ab_0    conda-forge
napari-crop               0.1.6              pyhd8ed1ab_0    conda-forge
napari-curtain            0.1.1              pyhd8ed1ab_0    conda-forge
napari-czifile2           0.2.7              pyhd8ed1ab_0    conda-forge
napari-folder-browser     0.1.3              pyhd8ed1ab_0    conda-forge
napari-layer-details-display 0.1.4              pyhd8ed1ab_0    conda-forge
napari-mouse-controls     0.1.3              pyhd8ed1ab_0    conda-forge
napari-plot-profile       0.2.1              pyhd8ed1ab_0    conda-forge
napari-plugin-engine      0.2.0              pyhd8ed1ab_2    conda-forge
napari-plugin-search      0.1.3              pyhd8ed1ab_0    conda-forge
napari-pyclesperanto-assistant 0.20.0             pyhd8ed1ab_0    conda-forge
napari-pystackreg         0.1.1              pyhd8ed1ab_0    conda-forge
napari-roi                0.1.7              pyhd8ed1ab_0    conda-forge
napari-segment-blobs-and-things-with-membranes 0.3.3              pyhd8ed1ab_0    conda-forge
napari-simpleitk-image-processing 0.4.4              pyhd8ed1ab_0    conda-forge
napari-skimage-regionprops 0.5.4              pyhd8ed1ab_0    conda-forge
napari-svg                0.1.6              pyhd8ed1ab_0    conda-forge
napari-tabu               0.1.5              pyhd8ed1ab_0    conda-forge
napari-time-slicer        0.4.9              pyhd8ed1ab_1    conda-forge
napari-tools-menu         0.1.17             pyhd8ed1ab_0    conda-forge
napari-workflow-inspector 0.2.2              pyhd8ed1ab_0    conda-forge
napari-workflow-optimizer 0.1.4              pyhd8ed1ab_0    conda-forge
napari-workflows          0.2.4              pyhd8ed1ab_0    conda-forge
natsort                   8.2.0              pyhd8ed1ab_0    conda-forge
nbclassic                 0.4.4              pyhd8ed1ab_0    conda-forge
nbclient                  0.6.8              pyhd8ed1ab_0    conda-forge
nbconvert                 7.0.0              pyhd8ed1ab_0    conda-forge
nbconvert-core            7.0.0              pyhd8ed1ab_0    conda-forge
nbconvert-pandoc          7.0.0              pyhd8ed1ab_0    conda-forge
nbformat                  5.6.1              pyhd8ed1ab_0    conda-forge
nest-asyncio              1.5.6              pyhd8ed1ab_0    conda-forge
networkx                  2.8.7              pyhd8ed1ab_0    conda-forge
notebook                  6.4.12             pyha770c72_0    conda-forge
notebook-shim             0.1.0              pyhd8ed1ab_0    conda-forge
npe2                      0.6.1              pyhd8ed1ab_0    conda-forge
numba                     0.56.2           py39h99ae161_1    conda-forge
numcodecs                 0.10.2           py39h415ef7b_0    conda-forge
numpy                     1.23.3           py39h9061af7_0    conda-forge
numpydoc                  1.4.0              pyhd8ed1ab_1    conda-forge
opencv                    4.6.0            py39hcbf5309_0    conda-forge
openh264                  2.3.1                h63175ca_1    conda-forge
openjpeg                  2.5.0                hc9384bd_1    conda-forge
openssl                   1.1.1q               h8ffe710_0    conda-forge
packaging                 21.3               pyhd8ed1ab_0    conda-forge
pandas                    1.5.0            py39h2ba5b7c_0    conda-forge
pandoc                    2.19.2               h57928b3_0    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
partd                     1.3.0              pyhd8ed1ab_0    conda-forge
pcre2                     10.37                hdfff0fc_1    conda-forge
pep517                    0.12.0           py39hcbf5309_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    9.2.0            py39hcef8f5f_2    conda-forge
pint                      0.19.2             pyhd8ed1ab_0    conda-forge
pip                       22.2.2             pyhd8ed1ab_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_0    conda-forge
platformdirs              2.5.2              pyhd8ed1ab_1    conda-forge
platymatch                0.0.3              pyhd8ed1ab_0    conda-forge
pluggy                    1.0.0            py39hcbf5309_3    conda-forge
ply                       3.11                       py_1    conda-forge
pooch                     1.6.0              pyhd8ed1ab_0    conda-forge
prometheus_client         0.14.1             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.31             pyha770c72_0    conda-forge
prompt_toolkit            3.0.31               hd8ed1ab_0    conda-forge
psutil                    5.9.2            py39ha55989b_0    conda-forge
psygnal                   0.3.5            py39h2e07f2f_0    conda-forge
pthread-stubs             0.4               hcd874cb_1001    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
py                        1.11.0             pyh6c4a22f_0    conda-forge
py-opencv                 4.6.0            py39h832f523_0    conda-forge
pyclesperanto-prototype   0.19.2             pyhd8ed1ab_0    conda-forge
pycodestyle               2.9.1              pyhd8ed1ab_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pydantic                  1.10.2           py39ha55989b_0    conda-forge
pygments                  2.13.0             pyhd8ed1ab_0    conda-forge
pynndescent               0.5.7              pyh6c4a22f_0    conda-forge
pyopencl                  2022.2.3         py39hdc8ea75_0    conda-forge
pyopengl                  3.1.6              pyhd8ed1ab_1    conda-forge
pyopenssl                 22.0.0             pyhd8ed1ab_1    conda-forge
pyparsing                 3.0.9              pyhd8ed1ab_0    conda-forge
pyperclip                 1.8.2              pyhd8ed1ab_2    conda-forge
pyqt                      5.15.7           py39hb08f45d_0    conda-forge
pyqt5-sip                 12.11.0          py39h415ef7b_0    conda-forge
pyqtgraph                 0.13.1             pyhd8ed1ab_0    conda-forge
pyrsistent                0.18.1           py39hb82d6ee_1    conda-forge
pysocks                   1.7.1              pyh0701188_6    conda-forge
pyspellchecker            0.7.0              pyhd8ed1ab_0    conda-forge
pystackreg                0.2.6.post1      py39h2e25243_0    conda-forge
pytest                    7.1.3            py39hcbf5309_0    conda-forge
python                    3.9.13          h9a09f29_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.16.2             pyhd8ed1ab_0    conda-forge
python_abi                3.9                      2_cp39    conda-forge
pytomlpp                  1.0.11           py39h1f6ef14_0    conda-forge
pytools                   2022.1.12          pyhd8ed1ab_0    conda-forge
pytorch                   1.12.1              py3.9_cpu_0    pytorch
pytorch-mutex             1.0                         cpu    pytorch
pytz                      2022.2.1           pyhd8ed1ab_0    conda-forge
pywavelets                1.3.0            py39h5d4886f_1    conda-forge
pywin32                   303              py39hb82d6ee_0    conda-forge
pywinpty                  2.0.8            py39h99910a6_0    conda-forge
pyyaml                    6.0              py39hb82d6ee_4    conda-forge
pyzmq                     24.0.1           py39hea35a22_0    conda-forge
qt-main                   5.15.6               hf0cf448_0    conda-forge
qtconsole                 5.3.2              pyhd8ed1ab_0    conda-forge
qtconsole-base            5.3.2              pyha770c72_0    conda-forge
qtpy                      2.2.0              pyhd8ed1ab_0    conda-forge
redlionfish               0.8              py39hcbf5309_0    conda-forge
reikna                    0.7.6              pyh6c4a22f_0    conda-forge
requests                  2.28.1             pyhd8ed1ab_1    conda-forge
rich                      12.5.1             pyhd8ed1ab_0    conda-forge
scikit-image              0.19.3           py39h2e25243_1    conda-forge
scikit-learn              1.1.2            py39hfd4428b_0    conda-forge
scipy                     1.9.1            py39h316f440_0    conda-forge
send2trash                1.8.0              pyhd8ed1ab_0    conda-forge
setuptools                65.4.0             pyhd8ed1ab_0    conda-forge
shellingham               1.5.0              pyhd8ed1ab_0    conda-forge
simpleitk                 2.1.0            py39hc8d2e25_1    conda-forge
sip                       6.6.2            py39h415ef7b_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sly                       0.4                pyhd8ed1ab_0    conda-forge
slycot                    0.5.0.0          py39h3ce7397_1    conda-forge
snappy                    1.1.9                h82413e6_1    conda-forge
sniffio                   1.3.0              pyhd8ed1ab_0    conda-forge
snowballstemmer           2.2.0              pyhd8ed1ab_0    conda-forge
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.3.2.post1        pyhd8ed1ab_0    conda-forge
sphinx                    5.2.3              pyhd8ed1ab_0    conda-forge
sphinxcontrib-applehelp   1.0.2                      py_0    conda-forge
sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
sphinxcontrib-htmlhelp    2.0.0              pyhd8ed1ab_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.5              pyhd8ed1ab_2    conda-forge
sqlite                    3.39.3               hcfcfb64_0    conda-forge
stack_data                0.5.1              pyhd8ed1ab_0    conda-forge
stackview                 0.3.5              pyhd8ed1ab_0    conda-forge
suitesparse               5.4.0                h5d0cbe0_1    conda-forge
superqt                   0.3.5              pyhd8ed1ab_0    conda-forge
svt-av1                   1.2.1                h0e60522_0    conda-forge
sympy                     1.11.1           py39hcbf5309_1    conda-forge
tabulate                  0.8.10             pyhd8ed1ab_0    conda-forge
tbb                       2021.6.0             h91493d7_0    conda-forge
tbb-devel                 2021.6.0             h91493d7_0    conda-forge
tblib                     1.7.0              pyhd8ed1ab_0    conda-forge
terminado                 0.16.0             pyh08f2357_0    conda-forge
the-segmentation-game     0.1.0              pyhd8ed1ab_0    conda-forge
threadpoolctl             3.1.0              pyh8a188c0_0    conda-forge
tifffile                  2022.8.12          pyhd8ed1ab_0    conda-forge
tinycss2                  1.1.1              pyhd8ed1ab_0    conda-forge
tk                        8.6.12               h8ffe710_0    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
tornado                   6.1              py39hb82d6ee_3    conda-forge
tqdm                      4.64.1             pyhd8ed1ab_0    conda-forge
traitlets                 5.4.0              pyhd8ed1ab_0    conda-forge
transforms3d              0.4.1              pyhd8ed1ab_0    conda-forge
typer                     0.6.1              pyhd8ed1ab_0    conda-forge
typing-extensions         4.3.0                hd8ed1ab_0    conda-forge
typing_extensions         4.3.0              pyha770c72_0    conda-forge
tzdata                    2022d                h191b570_0    conda-forge
ucrt                      10.0.20348.0         h57928b3_0    conda-forge
umap-learn                0.5.3            py39hcbf5309_0    conda-forge
unicodedata2              14.0.0           py39hb82d6ee_1    conda-forge
urllib3                   1.26.11            pyhd8ed1ab_0    conda-forge
vc                        14.2                 hb210afc_7    conda-forge
vispy                     0.9.6            py39h5d4886f_1    conda-forge
vs2015_runtime            14.29.30139          h890b9b1_7    conda-forge
wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
webencodings              0.5.1                      py_1    conda-forge
websocket-client          1.4.1              pyhd8ed1ab_0    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
widgetsnbextension        4.0.3              pyhd8ed1ab_0    conda-forge
win32_setctime            1.1.0              pyhd8ed1ab_0    conda-forge
win_inet_pton             1.1.0            py39hcbf5309_4    conda-forge
winpty                    0.4.3                         4    conda-forge
wrapt                     1.14.1           py39hb82d6ee_0    conda-forge
x264                      1!164.3095           h8ffe710_2    conda-forge
x265                      3.5                  h2d74725_3    conda-forge
xorg-libxau               1.0.9                hcd874cb_0    conda-forge
xorg-libxdmcp             1.1.3                hcd874cb_0    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
yaml                      0.2.5                h8ffe710_2    conda-forge
zarr                      2.13.2             pyhd8ed1ab_1    conda-forge
zeromq                    4.3.4                h0e60522_1    conda-forge
zfp                       1.0.0                h0e60522_1    conda-forge
zict                      2.2.0              pyhd8ed1ab_0    conda-forge
zipp                      3.8.1              pyhd8ed1ab_0    conda-forge
zlib-ng                   2.0.6                h8ffe710_0    conda-forge
zstd                      1.5.2                h7755175_4    conda-forge

I will remove cellpose-napari until this has been figured out.

haesleinhuepf commented 2 years ago

Related issue:

https://github.com/pytorch/pytorch/pull/7979

kevinyamauchi commented 2 years ago

Now that https://github.com/conda-forge/devbio-napari-feedstock/pull/17 has merged and @psobolewskiPhD has tested it, I think we can revisit this.

Based on this comment, I think we can specify a torch dependency and ask the user to add the pytorch channel to their installation command.

conda create -n napari-dev-bio python=3.9 devbio-napari -c pytorch -c conda-forge

When conda-forge adds support for specifying additional channels, we can add the pytorch channel to the recipe and remove the -c pytorch from the suggested installation command.

I would suggest that we start with something like pytorch 1.11.0. The 1.12 series doesn't install properly on Mac for me: https://github.com/haesleinhuepf/devbio-napari/issues/17#issuecomment-1251595341

psobolewskiPhD commented 2 years ago

FYI: on my arm64 macOS 12.6 mamba create --name test-env2 python=3.9 cellpose-napari JustWorks with + pytorch 1.12.1 cpu_py39hf1faf6a_1 being installed. Note: On my M1, 1.12.x is markedly faster than before because it's using Apple Accelerate library and the AMX for BLAS.

haesleinhuepf commented 2 years ago

JustWorks with + pytorch 1.12.1 cpu_py39hf1faf6a_1 being installed.

Do you remember how you installed that? That's the key problem ;-)

kevinyamauchi commented 2 years ago

FYI: on my arm64 macOS 12.6 mamba create --name test-env2 python=3.9 cellpose-napari JustWorks with + pytorch 1.12.1 cpu_py39hf1faf6a_1 being installed.

Thanks!

Note: On my M1, 1.12.x is markedly faster than before because it's using Apple Accelerate library and the AMX for BLAS.

Yeah, it would be great to use torch 1.12.x for the acceleration on M1/M2! However, currently with 1.12.1 there are issues with Intel Macs 🤦 : https://github.com/pytorch/pytorch/issues/78490#issuecomment-1217134036

For now, I think there are a couple of paths forward to including pytorch deps with devbio-napari (open to other suggestions!)

Include a specific pytorch as a run dependency in the conda-forge recipe. The advantage of this is that it could be a one line install that would work on all platforms. The downside is that we would probably want to specific pytorch==1.11.0 as pytorch 1.12.1 currently doesn't work on Intel Macs. However, this would create a conflict for M1/M2 Mac folks that want to use 1.12.1 to take advantage of the acceleration.
Ask the users to install pytorch themselves before installing devbio-napari. The advantage is that the user would be able to install the best version for their platform. The downside is that many users won't know how to figure out what the best version is for their platform.
wait for pytorch to release 1.12.2 and hope they fix the issue on Intel Macs (or update their conda recipe)
other ideas?

psobolewskiPhD commented 2 years ago

@kevinyamauchi What python env are you seeing cellpose issues on Intel macOS? It runs fine—not very fast, but fine—for me from conda-forge in rosetta. I just did mamba install cellpose-napari in an env with just napari. (Just conda-forge channel BTW.) pytorch 1.12.1 cpu_py39h9b0ea23_0 conda-forge

napari: 0.4.15
Platform: macOS-12.6-x86_64-i386-64bit
System: MacOS 12.6
Python: 3.9.12 | packaged by conda-forge | (main, Mar 24 2022, 23:23:20) [Clang 12.0.1 ]
Qt: 5.12.9
PyQt5: 5.12.3
NumPy: 1.22.3
SciPy: 1.8.0
Dask: 2022.04.0
VisPy: 0.9.6

OpenGL:
- GL version: 2.1 Metal - 76.3
- MAX_TEXTURE_SIZE: 16384

Screens:
- screen 1: resolution 1680x1050, scale 2.0

Plugins:
- console: 0.0.4
- scikit-image: 0.4.15
- svg: 0.1.5

Edit: i can't reproduce the issue in that thread either.

kevinyamauchi commented 2 years ago

I do the following:

conda create -n test-pytorch python=3.9 numpy pytorch -c pytorch -c conda-forge
conda activate test-pytorch
python -c "import numpy;import torch"

and I get the same error as linked in the issue above:

OMP: Error #15: Initializing libiomp5.dylib, but found libomp.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.
[1]    38426 abort      python -c "import numpy;import torch"

psobolewskiPhD commented 2 years ago

why not just: conda create -n test-pytorch python=3.9 pytorch -c conda-forge I think using the different channels mixing numpy and pytorch from different channels is problematic.

kevinyamauchi commented 2 years ago

why not just: conda create -n test-pytorch python=3.9 pytorch -c conda-forge I think using the different channels mixing numpy and pytorch from different channels is problematic.

Personally, I've always installed from the pytorch channel, as those are the official instructions on https://pytorch.org/. In the context of devbio-napari, as far as I know the conda-forge pytorch doesn't have a build for windows, so I think the pytorch channel is preferred.

I haven't had issues mixing numpy from conda-forge and pytorch from the pytorch channel before 1.12.0 - I think this is supposed to be supported given the discussion in the issue I linked above.

That being said, I tried installing pytorch from conda-forge on my Intel Mac as you suggested and it works!

kevinyamauchi commented 2 years ago

What if the installation instructions were:

conda create -n test-pytorch python=3.9 numpy pytorch  -c conda-forge -c pytorch

I think this makes conda-forge the higher priority channel so that it will install pytorch from conda-forge and if that fails (e.g., Windows), it will fall back to the pytorch channel. We should verify that the order channels are provided is indeed guaranteed to set the priority of the channels.

This works on my Intel Mac. I'd be curious if it also works on Windows (cc @haesleinhuepf ).

kevinyamauchi commented 2 years ago

Okay, I just checked the docs. It looks like the channels are indeed searched in the order they are given. Thus, I think providing both conda-forge and pytorch channels might be a good solution (assuming it works on Windows). Thanks for the tip about conda-forge channel, @psobolewskiPhD !

https://docs.conda.io/projects/conda/en/latest/commands/install.html#Channel%20Customization

psobolewskiPhD commented 2 years ago

Why specify numpy? It's a depend, it will come along?

kevinyamauchi commented 2 years ago

Why specify numpy? It's a depend, it will come along?

Sorry for being unclear. I was just trying to create a minimal environment for testing pytorch installation. The bug with pytorch 1.12.1 pops up when trying to import numpy and then pytorch.

edit: did you mean that numpy is a dependency of pytorch? If you install pytorch from the pytorch channel, it doesn't install numpy. So I think you need to specify numpy for the windows case since it will have to install from the pytorch channel (don't have a windows machine to test at the moment).

psobolewskiPhD commented 2 years ago

I meant in terms of devbio, because napari, cellpose, etc. definitely use numpy. But i get it now.

romainGuiet commented 1 year ago

Hi @kevinyamauchi , Hi @psobolewskiPhD , Hi @haesleinhuepf ,

I came here because of the cellpose title and I'm not sure if you're still looking for a solution...

I tried on my Windows machine to create an env with : conda create -n test-pytorch python=3.9 numpy pytorch -c conda-forge -c pytorch then conda activate test-pytorch and finally python -c "import numpy;import torch" and I got no error so far... and also tried python -c "import torch;print(torch.cuda.is_available())" which returns False as expected

conda list returns :

(test-pytorch_0) C:\Users\guiet>conda list
# packages in environment at C:\Users\guiet\.conda\envs\test-pytorch_0:
#
# Name                    Version                   Build  Channel
blas                      2.106                       mkl    conda-forge
bzip2                     1.0.8                h8ffe710_4    conda-forge
ca-certificates           2022.12.7            h5b45459_0    conda-forge
intel-openmp              2023.0.0         h57928b3_25922    conda-forge
libblas                   3.9.0                     6_mkl    conda-forge
libcblas                  3.9.0                     6_mkl    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
liblapack                 3.9.0                     6_mkl    conda-forge
liblapacke                3.9.0                     6_mkl    conda-forge
libsqlite                 3.40.0               hcfcfb64_0    conda-forge
libuv                     1.44.2               h8ffe710_0    conda-forge
libzlib                   1.2.13               hcfcfb64_4    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
mkl                       2020.4             hb70f87d_311    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
numpy                     1.24.2           py39h16ffa76_0    conda-forge
openssl                   3.0.8                hcfcfb64_0    conda-forge
pip                       23.0               pyhd8ed1ab_0    conda-forge
python                    3.9.16          h4de0772_0_cpython    conda-forge
python_abi                3.9                      3_cp39    conda-forge
pytorch                   1.13.1              py3.9_cpu_0    pytorch
pytorch-mutex             1.0                         cpu    pytorch
setuptools                67.1.0             pyhd8ed1ab_0    conda-forge
tk                        8.6.12               h8ffe710_0    conda-forge
typing_extensions         4.4.0              pyha770c72_0    conda-forge
tzdata                    2022g                h191b570_0    conda-forge
ucrt                      10.0.22621.0         h57928b3_0    conda-forge
vc                        14.3                hb6edc58_10    conda-forge
vs2015_runtime            14.34.31931         h4c5c07a_10    conda-forge
wheel                     0.38.4             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge

is there something else to try ? I'll be happy to see cellpose added to devbio !

Cheers,

Romain

haesleinhuepf commented 1 year ago

The issue might be resolved with this one:

https://github.com/BiAPoL/napari-clusters-plotter/issues/248

If so, we can add cellpose again to devbio-napari 🌞

kevinyamauchi commented 1 year ago

Super nice! Thanks, @haesleinhuepf !

haesleinhuepf / devbio-napari

issue with cellpose #17