[QST] Is it available for OS Win?

otakarkuchar commented 3 years ago

I have tried install in Conda prompt: pip install cucim ERROR: Could not find a version that satisfies the requirement cucim (from versions: none) ERROR: No matching distribution found for cucim

Is it available for OS Win or what I have made done wrong? Thank you for asist

quasiben commented 3 years ago

You should be able to install via PIP and it's possible it works with Windows. If you run into trouble you could try running RAPIDS on WSL, which may provide an alternative path forward . We just released 21.08 on pypi yesterday

haesleinhuepf commented 3 years ago

You should be able to install via PIP and it's possible it works with Windows

That sounds fantastic! However, when I run pip, it says:

 C:\structure\code\pyclesperanto_prototype>pip install cucim==21.08
ERROR: Could not find a version that satisfies the requirement cucim==21.08 (from versions: none)
ERROR: No matching distribution found for cucim==21.08

Any ideas how to debug this?

Btw. WSL is no way forward in my usecase unfortunately...

quasiben commented 3 years ago

On PyPi, I believe the version is 21.8.1. Running pip install cucim should be the latest version

otakarkuchar commented 3 years ago

On PyPi, I believe the version is 21.8.1. Running pip install cucim should be the latest version

When I did: pip install PyPi Successfully installed PyPi-2.1

pip install cucim ERROR: Could not find a version that satisfies the requirement cucim (from versions: none) ERROR: No matching distribution found for cucim

Can you please describe the installation procedure steps? thank you so much

gigony commented 3 years ago

Hi @otakarkuchar,

Since the image loader part(cucim.clara module) of cuCIM supports (and assumed) Linux environment only, the current PyPI package (wheel[.whl] file) doesn't support Windows (although it supports WSL) for now.

Could you tell us your use case with cuCIM? If you only needs image processing API based on CuPy (cucim.skimage module which is pure python code), I think it is possible that we may be able to release a PyPI package for Windows, having only cucim.skimage module in the next release.

otakarkuchar commented 3 years ago

Hi @otakarkuchar,

Since the image loader part(cucim.clara module) of cuCIM supports (and assumed) Linux environment only, the current PyPI package (wheel[.whl] file) doesn't support Windows (although it supports WSL) for now.

Could you tell us your use case with cuCIM? If you only needs image processing API based on CuPy (cucim.skimage module which is pure python code), I think it is possible that we may be able to release a PyPI package for Windows, having only cucim.skimage module in the next release.

Yes, my case is only image processing on 3D datasets. If you do that, it will be great! I am looking forward to try cucim.skimage library! ... Thank you guys!

haesleinhuepf commented 3 years ago

I think it is possible that we may be able to release a PyPI package for Windows, having only cucim.skimage module in the next release.

Oh that would be great indeed! ❤️

grlee77 commented 3 years ago

Although not officially documented, as long as you have CuPy >= 9.0 installed you should be able to install cucim.skimage standalone (without the need to compile any code) as described here: https://github.com/rapidsai/cucim/issues/64#issuecomment-878535597

haesleinhuepf commented 3 years ago

Although not officially documented, as long as you have CuPy >= 9.0 installed you should be able to install cucim.skimage standalone (without the need to compile any code) as described here: #64 (comment)

That's great! Are there any plans for making an independent package out of that? Would be great to pip install cucim-scikit-image. ;-)

grlee77 commented 3 years ago

Are there any plans for making an independent package out of that?

I will try to discuss with @gigony and the team next week about what potential options for this might be across pip and conda installs. I suppose we could make a cucim_skimage conda package that installed as described in my prior comment as long as we mark that package in conflict with the main cucim one so that users can only install one or the other, but not both. A second option is splitting the cucim package into separate cucim.clara and cucim.skimage namespace packages, but I would like to avoid that if possible.

gigony commented 3 years ago

Hi @otakarkuchar and @haesleinhuepf ,

The team had a discussion yesterday regarding this issue (providing a PyPI windows package).

Summary

Since we don't have an infra to build & test a cuCIM PyPI package on Windows and the team currently is focusing on implementing features on Linux platform, we wouldn't want to officially release the Windows PyPI package for now.
Instead, we would like to provide a guideline to install cuCIM (only scikit-image part) on Windows, hoping that it would solve the user's use case.
- pip install -e "git+https://github.com/rapidsai/cucim.git@<tag(version)>#egg=cucim&subdirectory=python/cucim" scipy scikit-image
- e.g, pip install -e "git+https://github.com/rapidsai/cucim.git@v21.08.01#egg=cucim&subdirectory=python/cucim" scipy scikit-image

Background

cuCIM is part of RAPIDS project and all RAPIDS projects (cuDF, cuSignal, cuGraph, cuML, etc) except cuCIM are focusing on the Linux platform only and dropped the support for PyPI since RAPIDS v0.7.

cuCIM is exceptionally providing a Linux PyPI package separately by NVIDIA Clara team to support MONAI and Clara Train use case.

Since all RAPIDS projects depend on GPU CI to build and release a conda package, we don't have a good infra to build/test a conda package on Windows platform (We had a Windows-specific bug before ).

For this reason, we have decided to focus on the Linux Platform and implementing other requested features for now. Instead, we would like to give you a workaround solution to install cuCIM on Windows.

Instruction to install cuCIM on Windows

After cupy is installed (via conda or pip: See https://docs.cupy.dev/en/stable/install.html), please execute the following command (double-quote(") is necessary on Windows):

pip install -e "git+https://github.com/rapidsai/cucim.git@v21.08.01#egg=cucim&subdirectory=python/cucim" scipy scikit-image

In above command, v21.08.01 is a tag name available here.

I have tested it on my personal machine and it worked:

conda create -n cucim -c conda-forge python=3.8 cudatoolkit=11.0 cupy cudnn cutensor
conda activate cucim
pip install -e "git+https://github.com/rapidsai/cucim.git@v21.08.01#egg=cucim&subdirectory=python/cucim" scipy scikit-image

You can also install cuCIM thorough requirements.txt file

=requirement.txt=

-e "git+https://github.com/rapidsai/cucim.git@v21.08.01#egg=cucim&subdirectory=python/cucim"
scipy
scikit-image

= In terminal =

pip install -r requirements.txt

We are sorry that we couldn't provide PyPI package for Windows. we hope this solution resolves your use cases.

Thanks!

Click to see detailed commands

(base) PS C:\Users\gigon> conda create -n cucim -c conda-forge python=3.8 cudatoolkit=11.0 cupy cudnn cutensor Collecting package metadata (current_repodata.json): done Solving environment: done ==> WARNING: A newer version of conda exists. <== current version: 4.10.1 latest version: 4.10.3 Please update conda by running $ conda update -n base -c defaults conda ## Package Plan ## environment location: C:\Users\gigon\anaconda3\envs\cucim added / updated specs: - cudatoolkit=11.0 - cudnn - cupy - cutensor - python=3.8 The following packages will be downloaded: package | build ---------------------------|----------------- cupy-9.3.0 | py38h66fa806_0 49.3 MB conda-forge fastrlock-0.6 | py38h885f38d_1 33 KB conda-forge numpy-1.21.2 | py38h089cfbf_0 5.6 MB conda-forge python-3.8.10 |h7840368_1_cpython 19.3 MB conda-forge setuptools-57.4.0 | py38haa244fe_0 968 KB conda-forge ------------------------------------------------------------ Total: 75.2 MB The following NEW packages will be INSTALLED: ca-certificates conda-forge/win-64::ca-certificates-2021.5.30-h5b45459_0 cudatoolkit conda-forge/win-64::cudatoolkit-11.0.3-h3f58a73_8 cudnn conda-forge/win-64::cudnn-8.2.1.32-h754d62a_0 cupy conda-forge/win-64::cupy-9.3.0-py38h66fa806_0 cutensor conda-forge/win-64::cutensor-1.3.1.3-h5987435_2 fastrlock conda-forge/win-64::fastrlock-0.6-py38h885f38d_1 intel-openmp conda-forge/win-64::intel-openmp-2021.3.0-h57928b3_3372 libblas conda-forge/win-64::libblas-3.9.0-11_win64_mkl libcblas conda-forge/win-64::libcblas-3.9.0-11_win64_mkl liblapack conda-forge/win-64::liblapack-3.9.0-11_win64_mkl mkl conda-forge/win-64::mkl-2021.3.0-hb70f87d_564 numpy conda-forge/win-64::numpy-1.21.2-py38h089cfbf_0 openssl conda-forge/win-64::openssl-1.1.1k-h8ffe710_1 pip conda-forge/noarch::pip-21.2.4-pyhd8ed1ab_0 python conda-forge/win-64::python-3.8.10-h7840368_1_cpython python_abi conda-forge/win-64::python_abi-3.8-2_cp38 setuptools conda-forge/win-64::setuptools-57.4.0-py38haa244fe_0 sqlite conda-forge/win-64::sqlite-3.36.0-h8ffe710_0 tbb conda-forge/win-64::tbb-2021.3.0-h2d74725_0 ucrt conda-forge/win-64::ucrt-10.0.20348.0-h57928b3_0 vc conda-forge/win-64::vc-14.2-hb210afc_5 vs2015_runtime conda-forge/win-64::vs2015_runtime-14.29.30037-h902a5da_5 wheel conda-forge/noarch::wheel-0.37.0-pyhd8ed1ab_1 Proceed ([y]/n)? y Downloading and Extracting Packages cupy-9.3.0 | 49.3 MB | ############################################################################################ | 100% fastrlock-0.6 | 33 KB | ############################################################################################ | 100% numpy-1.21.2 | 5.6 MB | ############################################################################################ | 100% python-3.8.10 | 19.3 MB | ############################################################################################ | 100% setuptools-57.4.0 | 968 KB | ############################################################################################ | 100% Preparing transaction: done Verifying transaction: done Executing transaction: / "By downloading and using the CUDA Toolkit conda packages, you accept the terms and conditions of the CUDA End User License Agreement (EULA): https://docs.nvidia.com/cuda/eula/index.html" | "By downloading and using the cuDNN conda packages, you accept the terms and conditions of the NVIDIA cuDNN EULA - https://docs.nvidia.com/deeplearning/cudnn/sla/index.html" | "By downloading and using the cuTENSOR conda package, you accept the terms" "and conditions of the NVIDIA cuTENSOR Software License Agreement:" "https://docs.nvidia.com/cuda/cutensor/license.html" done # # To activate this environment, use # # $ conda activate cucim # # To deactivate an active environment, use # # $ conda deactivate (base) PS C:\Users\gigon> conda activate cucim (cucim) PS C:\Users\gigon> pip install -e "git+https://github.com/rapidsai/cucim.git@v21.08.01#egg=cucim&subdirectory=python/cucim" scipy scikit-image Obtaining cucim from git+https://github.com/rapidsai/cucim.git@v21.08.01#egg=cucim&subdirectory=python/cucim Updating c:\users\gigon\src\cucim clone (to revision v21.08.01) Running command git fetch -q --tags Running command git reset --hard -q 12f0c05957358cf83e7c2be0150dda7cccc37160 Collecting click Using cached click-8.0.1-py3-none-any.whl (97 kB) Requirement already satisfied: numpy in c:\users\gigon\anaconda3\envs\cucim\lib\site-packages (from cucim) (1.21.2) Collecting colorama Using cached colorama-0.4.4-py2.py3-none-any.whl (16 kB) Installing collected packages: colorama, click, cucim Running setup.py develop for cucim Collecting scipy Downloading scipy-1.7.1-cp38-cp38-win_amd64.whl (33.7 MB) |████████████████████████████████| 33.7 MB 6.4 MB/s Collecting scikit-image Downloading scikit_image-0.18.3-cp38-cp38-win_amd64.whl (12.2 MB) |████████████████████████████████| 12.2 MB 6.4 MB/s Requirement already satisfied: numpy<1.23.0,>=1.16.5 in c:\users\gigon\anaconda3\envs\cucim\lib\site-packages (from scipy) (1.21.2) Collecting networkx>=2.0 Downloading networkx-2.6.2-py3-none-any.whl (1.9 MB) |████████████████████████████████| 1.9 MB 6.8 MB/s Collecting imageio>=2.3.0 Downloading imageio-2.9.0-py3-none-any.whl (3.3 MB) |████████████████████████████████| 3.3 MB ... Collecting pillow!=7.1.0,!=7.1.1,>=4.3.0 Downloading Pillow-8.3.1-1-cp38-cp38-win_amd64.whl (3.2 MB) |████████████████████████████████| 3.2 MB 6.4 MB/s Collecting matplotlib!=3.0.0,>=2.0.0 Downloading matplotlib-3.4.3-cp38-cp38-win_amd64.whl (7.1 MB) |████████████████████████████████| 7.1 MB 6.4 MB/s Collecting tifffile>=2019.7.26 Downloading tifffile-2021.8.8-py3-none-any.whl (171 kB) |████████████████████████████████| 171 kB ... Collecting PyWavelets>=1.1.1 Downloading PyWavelets-1.1.1-cp38-cp38-win_amd64.whl (4.3 MB) |████████████████████████████████| 4.3 MB 6.8 MB/s Requirement already satisfied: click in c:\users\gigon\anaconda3\envs\cucim\lib\site-packages (from cucim) (8.0.1) Collecting kiwisolver>=1.0.1 Downloading kiwisolver-1.3.1-cp38-cp38-win_amd64.whl (51 kB) |████████████████████████████████| 51 kB ... Collecting pyparsing>=2.2.1 Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB) Collecting cycler>=0.10 Downloading cycler-0.10.0-py2.py3-none-any.whl (6.5 kB) Collecting python-dateutil>=2.7 Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB) |████████████████████████████████| 247 kB ... Collecting six Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) Requirement already satisfied: colorama in c:\users\gigon\anaconda3\envs\cucim\lib\site-packages (from click->cucim) (0.4.4) Installing collected packages: six, python-dateutil, pyparsing, pillow, kiwisolver, cycler, tifffile, scipy, PyWavelets, networkx, matplotlib, imageio, scikit-image, cucim Attempting uninstall: cucim Found existing installation: cucim 21.8.1 Uninstalling cucim-21.8.1: Successfully uninstalled cucim-21.8.1 Running setup.py develop for cucim Successfully installed PyWavelets-1.1.1 cucim-21.8.1 cycler-0.10.0 imageio-2.9.0 kiwisolver-1.3.1 matplotlib-3.4.3 networkx-2.6.2 pillow-8.3.1 pyparsing-2.4.7 python-dateutil-2.8.2 scikit-image-0.18.3 scipy-1.7.1 six-1.16.0 tifffile-2021.8.8 (cucim) PS C:\Users\gigon> python Python 3.8.10 | packaged by conda-forge | (default, May 11 2021, 06:25:23) [MSC v.1916 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> from cucim.skimage import color C:\Users\gigon\anaconda3\envs\cucim\lib\site-packages\cupy\_environment.py:213: UserWarning: CUDA path could not be detected. Set CUDA_PATH environment variable if CuPy fails to load. warnings.warn( >>> dir(color) ['__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'ahx_from_rgb', 'bex_from_rgb', 'bpx_from_rgb', 'bro_from_rgb', 'color_dict', 'colorconv', 'colorlabel', 'combine_stains', 'convert_colorspace', 'deltaE_cie76', 'deltaE_ciede2000', 'deltaE_ciede94', 'deltaE_cmc', 'delta_e', 'fgx_from_rgb', 'gdx_from_rgb', 'gray2rgb', 'gray2rgba', 'grey2rgb', 'hax_from_rgb', 'hdx_from_rgb', 'hed2rgb', 'hed_from_rgb', 'hpx_from_rgb', 'hsv2rgb', 'lab2lch', 'lab2rgb', 'lab2xyz', 'label2rgb', 'lch2lab', 'luv2rgb', 'luv2xyz', 'rbd_from_rgb', 'rgb2gray', 'rgb2grey', 'rgb2hed', 'rgb2hsv', 'rgb2lab', 'rgb2luv', 'rgb2rgbcie', 'rgb2xyz', 'rgb2ycbcr', 'rgb2ydbdr', 'rgb2yiq', 'rgb2ypbpr', 'rgb2yuv', 'rgb_colors', 'rgb_from_ahx', 'rgb_from_bex', 'rgb_from_bpx', 'rgb_from_bro', 'rgb_from_fgx', 'rgb_from_gdx', 'rgb_from_hax', 'rgb_from_hdx', 'rgb_from_hed', 'rgb_from_hpx', 'rgb_from_rbd', 'rgba2rgb', 'rgbcie2rgb', 'separate_stains', 'xyz2lab', 'xyz2luv', 'xyz2rgb', 'ycbcr2rgb', 'ydbdr2rgb', 'yiq2rgb', 'ypbpr2rgb', 'yuv2rgb'] >>> >>> import cupy as cp >>> from cucim.skimage.color import rgb2gray >>> from skimage import data >>> img = cp.array(data.astronaut()) >>> img_gray = rgb2gray(img) >>> img_gray array([[5.83434902e-01, 4.14859216e-01, 2.44058431e-01, ..., 4.75007843e-01, 4.58213333e-01, 4.69121961e-01], [6.75588235e-01, 5.56006667e-01, 4.49052941e-01, ..., 4.68548627e-01, 4.56501176e-01, 4.55958431e-01], [7.66334902e-01, 7.00524314e-01, 6.49276078e-01, ..., 4.76406667e-01, 4.62104314e-01, 4.53978431e-01], ..., [6.81696471e-01, 6.81979216e-01, 6.71889020e-01, ..., 0.00000000e+00, 2.82745098e-04, 0.00000000e+00], [6.74694510e-01, 6.68532941e-01, 6.64030196e-01, ..., 2.82745098e-04, 3.92156863e-03, 0.00000000e+00], [6.70482353e-01, 6.63189804e-01, 6.52838824e-01, ..., 0.00000000e+00, 3.92156863e-03, 0.00000000e+00]])

haesleinhuepf commented 3 years ago

Thanks for your feedback @gigony !

We are sorry that we couldn't provide PyPI package for Windows. we hope this solution resolves your use cases.

I was hoping to make cuCIM available to end-users without coding skills via the napari platform (see also). But I see the point. I also struggle with multi-platform build / CI infrastructure when using GPUs. If you ever come to the point where you want to reach more end-users, e.g. using Windows, please get in touch! :-)

grlee77 commented 3 years ago

@haesleinhuepf, indeed it would be very nice to provide access to cuCIM functions through napari. This can potentially reach a wide audience of users who would not otherwise use the package directly.

Ideally, longer term, scikit-image itself will be able to dispatch to a cuCIM backend similar to how scipy.fft can already be configured to use CuPy as a backend. SciPy is in the process of extending that approach to the ndimage, linalg and special modules as well (ndimage has been completed and is under review).

One issue with the initial uarray-based FFT backends was that users had to explicitly set a global backend or use a context manager. However, we recently changed the default global backend to be tried last. That way a user can call register_backend once and the registered backend would always be tried first (see example at the top of that issue).

I have been testing a similar backend approach for skimage.filterswith intention to eventually have cuCIM as a backend option (no PR open yet for this).

This is getting a ways off topic for the issue here, but I am happy to discuss more either in another issue (or possibly at https://skimage.zulipchat.com or in a community call with the scikit-image team).

iperov commented 3 years ago

@gigony

Instruction to install cuCIM on Windows After cupy is installed

then why need cuCIM if CuPy should be installed?

gigony commented 3 years ago

Hello @iperov

Thanks for the question!

cuCIM's scikit-image API (image processing module based on CuPy) comes from @grlee77 (co-author of cuCIM)'s project named cupyimg which extends CuPy with additional functions for image/signal processing. With cuCIM, you can use other image processing algorithms that are not in CuPy's cupyx.scipy.ndimage.

Since pip install cucim would build&install from the source which takes a long time, CuPy(https://cupy.dev/) provides pre-built packages for each CUDA Toolkit version(such as cupy-cuda110). In addition, since users may want to use only data loader part (cucim.CuImage or cucim.clara which doesn't require CuPy) or just clara.skimage module, we made cupy not a mandatory package to install (please see setup.py).

When import cucim is executed, it first tries to import cupy and import cupy.Image (requires CUDA >= 11.0) optionally (please see __init__.py.

Please let me know if you have other questions. Thank you!

iperov commented 3 years ago

I don't like such a Game of Dependencies. I want to see cuCIM lib that will implement all cupy + gpu image processing only with cuda lib, without dependencies. Notify me when it happens.

gigony commented 3 years ago

Thanks @iperov for the feedback! We have a plan to implement some image processing algorithms in C++ that don't depend on CuPy but implementing all the image processing algorithms are not feasible and would take much time to implement. For this reason, we are also thinking about providing existing GPU-accelerated image processing implementations from 3rdparty libraries as a plugin. Arrayfire is one of the candidates. Please take a look at the library (it has a python wrapper, instruction) and it would be great if you could share your experience with it in terms of usability/functionality that is missing. Thank you!

grlee77 commented 3 years ago

I want to see cuCIM lib that will implement all cupy + gpu image processing only with cuda lib, without dependencies.

As you are likely aware, if you are working with 2D images, there is NVIDIA's NPP. It is a free CUDA library, but is not open source.

gigony commented 3 years ago

Closing this issue. Please feel free to reopen if you have any other questions. Thank you!

rapidsai / cucim