search

juglab / n2v

This is the implementation of Noise2Void training.
Other
385 stars 107 forks source link

Tensorflow-gpu version 2.4.1 is not compatile with Windows #111

Closed nunopimpaomartins closed 1 year ago

nunopimpaomartins commented 2 years ago

Just to report that the tensorflow gpu version written in the README instructions is not compatible with Windows machines. It is a Linux only version, from what I could check.

Might be a problem for new users when creating new environments in different OS.

Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - tensorflow-gpu=2.4.1

Might be good to test with newer versions of tensorflow-gpu and/or keras. I did not check the Keras version compatibility.

Nuno

cberri commented 2 years ago

Hey,

In my case I used (with CUDA 11.0 on Windows10):

​ pip install tensorflow-gpu==2.4.0 keras==2.3.1

I think conda does not "support" yet tensorflow-gpu=2.4.0 and keras=2.3.1

I hope it helps.

Carlo

Few more details: On Windows10 conda search results for TF and keras

(base) C:\Users\...>conda search tensorflow
Loading channels: done

# Name                       Version           Build  Channel

tensorflow                     1.7.0               0  pkgs/main
tensorflow                     1.7.0               0  anaconda
tensorflow                     1.7.1               0  pkgs/main
tensorflow                     1.7.1               0  anaconda
tensorflow                     1.8.0               0  pkgs/main
tensorflow                     1.8.0               0  anaconda
tensorflow                     1.9.0 eigen_py35hb0e21f4_1  pkgs/main
tensorflow                     1.9.0 eigen_py35hb0e21f4_1  anaconda
tensorflow                     1.9.0 eigen_py36h0b764b7_1  pkgs/main
tensorflow                     1.9.0 eigen_py36h0b764b7_1  anaconda
tensorflow                     1.9.0 gpu_py35h0075c17_1  pkgs/main
tensorflow                     1.9.0 gpu_py35h0075c17_1  anaconda
tensorflow                     1.9.0 gpu_py36hfdee9c2_1  pkgs/main
tensorflow                     1.9.0 gpu_py36hfdee9c2_1  anaconda
tensorflow                    1.10.0 eigen_py35h38c8211_0  pkgs/main
tensorflow                    1.10.0 eigen_py35h38c8211_0  anaconda
tensorflow                    1.10.0 eigen_py36h849fbd8_0  pkgs/main
tensorflow                    1.10.0 eigen_py36h849fbd8_0  anaconda
tensorflow                    1.10.0 gpu_py35ha5d5ef7_0  pkgs/main
tensorflow                    1.10.0 gpu_py35ha5d5ef7_0  anaconda
tensorflow                    1.10.0 gpu_py36h3514669_0  pkgs/main
tensorflow                    1.10.0 gpu_py36h3514669_0  anaconda
tensorflow                    1.10.0 mkl_py35h4a0f5c2_0  pkgs/main
tensorflow                    1.10.0 mkl_py35h4a0f5c2_0  anaconda
tensorflow                    1.10.0 mkl_py36hb361250_0  pkgs/main
tensorflow                    1.10.0 mkl_py36hb361250_0  anaconda
tensorflow                    1.11.0 eigen_py36h346fd36_0  pkgs/main
tensorflow                    1.11.0 eigen_py36h346fd36_0  anaconda
tensorflow                    1.11.0 gpu_py36h5dc63e2_0  pkgs/main
tensorflow                    1.11.0 gpu_py36h5dc63e2_0  anaconda
tensorflow                    1.11.0 mkl_py36h41bbc20_0  pkgs/main
tensorflow                    1.11.0 mkl_py36h41bbc20_0  anaconda
tensorflow                    1.12.0 eigen_py36h67ac661_0  pkgs/main
tensorflow                    1.12.0 eigen_py36h67ac661_0  anaconda
tensorflow                    1.12.0 gpu_py36ha5f9131_0  pkgs/main
tensorflow                    1.12.0 gpu_py36ha5f9131_0  anaconda
tensorflow                    1.12.0 mkl_py36h4f00353_0  pkgs/main
tensorflow                    1.12.0 mkl_py36h4f00353_0  anaconda
tensorflow                    1.13.1 eigen_py36hf0a88a9_0  pkgs/main
tensorflow                    1.13.1 eigen_py36hf0a88a9_0  anaconda
tensorflow                    1.13.1 eigen_py37h2a8d240_0  pkgs/main
tensorflow                    1.13.1 eigen_py37h2a8d240_0  anaconda
tensorflow                    1.13.1 gpu_py36h1635174_0  pkgs/main
tensorflow                    1.13.1 gpu_py36h1635174_0  anaconda
tensorflow                    1.13.1 gpu_py36h9006a92_0  pkgs/main
tensorflow                    1.13.1 gpu_py36h9006a92_0  anaconda
tensorflow                    1.13.1 gpu_py37h83e5d6a_0  pkgs/main
tensorflow                    1.13.1 gpu_py37h83e5d6a_0  anaconda
tensorflow                    1.13.1 gpu_py37hbc1a9d5_0  pkgs/main
tensorflow                    1.13.1 gpu_py37hbc1a9d5_0  anaconda
tensorflow                    1.13.1 mkl_py36hd212fbe_0  pkgs/main
tensorflow                    1.13.1 mkl_py36hd212fbe_0  anaconda
tensorflow                    1.13.1 mkl_py37h9463c59_0  pkgs/main
tensorflow                    1.13.1 mkl_py37h9463c59_0  anaconda
tensorflow                    1.14.0 eigen_py36hf4fd08c_0  pkgs/main
tensorflow                    1.14.0 eigen_py36hf4fd08c_0  anaconda
tensorflow                    1.14.0 eigen_py37hcf3f253_0  pkgs/main
tensorflow                    1.14.0 eigen_py37hcf3f253_0  anaconda
tensorflow                    1.14.0 gpu_py36h305fd99_0  pkgs/main
tensorflow                    1.14.0 gpu_py36h305fd99_0  anaconda
tensorflow                    1.14.0 gpu_py36heb2afb7_0  pkgs/main
tensorflow                    1.14.0 gpu_py36heb2afb7_0  anaconda
tensorflow                    1.14.0 gpu_py37h2fabf85_0  pkgs/main
tensorflow                    1.14.0 gpu_py37h2fabf85_0  anaconda
tensorflow                    1.14.0 gpu_py37h5512b17_0  pkgs/main
tensorflow                    1.14.0 gpu_py37h5512b17_0  anaconda
tensorflow                    1.14.0 mkl_py36hb88db5b_0  pkgs/main
tensorflow                    1.14.0 mkl_py36hb88db5b_0  anaconda
tensorflow                    1.14.0 mkl_py37h7908ca0_0  pkgs/main
tensorflow                    1.14.0 mkl_py37h7908ca0_0  anaconda
tensorflow                    1.15.0 eigen_py36h932cce6_0  pkgs/main
tensorflow                    1.15.0 eigen_py36h932cce6_0  anaconda
tensorflow                    1.15.0 eigen_py37h9f89a44_0  pkgs/main
tensorflow                    1.15.0 eigen_py37h9f89a44_0  anaconda
tensorflow                    1.15.0 gpu_py36h2b26d6b_0  pkgs/main
tensorflow                    1.15.0 gpu_py36h2b26d6b_0  anaconda
tensorflow                    1.15.0 gpu_py37hc3743a6_0  pkgs/main
tensorflow                    1.15.0 gpu_py37hc3743a6_0  anaconda
tensorflow                    1.15.0 mkl_py36h997801b_0  pkgs/main
tensorflow                    1.15.0 mkl_py36h997801b_0  anaconda
tensorflow                    1.15.0 mkl_py37h3789bd0_0  pkgs/main
tensorflow                    1.15.0 mkl_py37h3789bd0_0  anaconda
tensorflow                     2.0.0 eigen_py36h457aea3_0  pkgs/main
tensorflow                     2.0.0 eigen_py36h457aea3_0  anaconda
tensorflow                     2.0.0 eigen_py37hbfc5123_0  pkgs/main
tensorflow                     2.0.0 eigen_py37hbfc5123_0  anaconda
tensorflow                     2.0.0 gpu_py36hfdd5754_0  pkgs/main
tensorflow                     2.0.0 gpu_py36hfdd5754_0  anaconda
tensorflow                     2.0.0 gpu_py37h57d29ca_0  pkgs/main
tensorflow                     2.0.0 gpu_py37h57d29ca_0  anaconda
tensorflow                     2.0.0 mkl_py36h781710d_0  pkgs/main
tensorflow                     2.0.0 mkl_py36h781710d_0  anaconda
tensorflow                     2.0.0 mkl_py37he1bbcac_0  pkgs/main
tensorflow                     2.0.0 mkl_py37he1bbcac_0  anaconda
tensorflow                     2.1.0 eigen_py36hdbbabfe_0  pkgs/main
tensorflow                     2.1.0 eigen_py36hdbbabfe_0  anaconda
tensorflow                     2.1.0 eigen_py37hd727fc0_0  pkgs/main
tensorflow                     2.1.0 eigen_py37hd727fc0_0  anaconda
tensorflow                     2.1.0 gpu_py36h3346743_0  pkgs/main
tensorflow                     2.1.0 gpu_py36h3346743_0  anaconda
tensorflow                     2.1.0 gpu_py37h7db9008_0  pkgs/main
tensorflow                     2.1.0 gpu_py37h7db9008_0  anaconda
tensorflow                     2.1.0 mkl_py36h31ad7c1_0  pkgs/main
tensorflow                     2.1.0 mkl_py36h31ad7c1_0  anaconda
tensorflow                     2.1.0 mkl_py37ha977152_0  pkgs/main
tensorflow                     2.1.0 mkl_py37ha977152_0  anaconda
tensorflow                     2.3.0 mkl_py37h04bc1aa_0  pkgs/main
tensorflow                     2.3.0 mkl_py37h10aaca4_0  pkgs/main
tensorflow                     2.3.0 mkl_py37h3bad0a6_0  pkgs/main
tensorflow                     2.3.0 mkl_py37h48e11e3_0  pkgs/main
tensorflow                     2.3.0 mkl_py37h856240d_0  pkgs/main
tensorflow                     2.3.0 mkl_py37h936c3e2_0  pkgs/main
tensorflow                     2.3.0 mkl_py37h952ae9f_0  pkgs/main
tensorflow                     2.3.0 mkl_py37he40ee82_0  pkgs/main
tensorflow                     2.3.0 mkl_py37he70e3f7_0  pkgs/main
tensorflow                     2.3.0 mkl_py38h1fcfbd6_0  pkgs/main
tensorflow                     2.3.0 mkl_py38h37f7ee5_0  pkgs/main
tensorflow                     2.3.0 mkl_py38h3c6dea5_0  pkgs/main
tensorflow                     2.3.0 mkl_py38h46e32b0_0  pkgs/main
tensorflow                     2.3.0 mkl_py38h637f690_0  pkgs/main
tensorflow                     2.3.0 mkl_py38h8557ec7_0  pkgs/main
tensorflow                     2.3.0 mkl_py38h8c0d9a2_0  pkgs/main
tensorflow                     2.3.0 mkl_py38ha39cb68_0  pkgs/main
tensorflow                     2.3.0 mkl_py38hd19cc29_0  pkgs/main
tensorflow                     2.5.0 eigen_py37hba85c30_0  pkgs/main
tensorflow                     2.5.0 eigen_py38h6b3c56f_0  pkgs/main
tensorflow                     2.5.0 eigen_py39hf7bd2bc_0  pkgs/main
tensorflow                     2.5.0 gpu_py37h23de114_0  pkgs/main
tensorflow                     2.5.0 gpu_py38h8e8c102_0  pkgs/main
tensorflow                     2.5.0 gpu_py39h7dc34a2_0  pkgs/main
tensorflow                     2.5.0 mkl_py37h99b934d_0  pkgs/main
tensorflow                     2.5.0 mkl_py38hbe2df88_0  pkgs/main
tensorflow                     2.5.0 mkl_py39h1fa1df6_0  pkgs/main

(base) C:\Users\...>conda search keras
Loading channels: done

# Name                       Version           Build  Channel

keras                          2.0.8  py35h15001cb_0  pkgs/main
keras                          2.0.8  py35h15001cb_0  anaconda
keras                          2.0.8  py36h65e7a35_0  pkgs/main
keras                          2.0.8  py36h65e7a35_0  anaconda
keras                          2.1.2          py35_0  pkgs/main
keras                          2.1.2          py35_0  anaconda
keras                          2.1.2          py36_0  pkgs/main
keras                          2.1.2          py36_0  anaconda
keras                          2.1.3          py35_0  pkgs/main
keras                          2.1.3          py35_0  anaconda
keras                          2.1.3          py36_0  pkgs/main
keras                          2.1.3          py36_0  anaconda
keras                          2.1.4          py35_0  pkgs/main
keras                          2.1.4          py35_0  anaconda
keras                          2.1.4          py36_0  pkgs/main
keras                          2.1.4          py36_0  anaconda
keras                          2.1.5          py35_0  pkgs/main
keras                          2.1.5          py35_0  anaconda
keras                          2.1.5          py36_0  pkgs/main
keras                          2.1.5          py36_0  anaconda
keras                          2.1.6          py35_0  pkgs/main
keras                          2.1.6          py35_0  anaconda
keras                          2.1.6          py36_0  pkgs/main
keras                          2.1.6          py36_0  anaconda
keras                          2.2.0               0  pkgs/main
keras                          2.2.0               0  anaconda
keras                          2.2.2               0  pkgs/main
keras                          2.2.2               0  anaconda
keras                          2.2.4               0  pkgs/main
keras                          2.2.4               0  anaconda
keras                          2.3.1               0  pkgs/main
keras                          2.3.1               0  anaconda
keras                          2.4.3               0  pkgs/main
keras                          2.4.3               0  anaconda
zoccoler commented 2 years ago

Hi all,

I was getting the same error (in a fresh conda environment) and I am glad there is already an issue about it here.

Thus, would the developers recommend any specific version for windows with conda? If there is not, maybe a quick warning in the readme.md file would be helpful for windows users :)

I tried with tf-gpu 2.3.0 and 2.5.0 but got this same error:

Pinned packages:
  - python 3.7.*

Encountered problems while solving:
  - package tensorflow-gpu-2.5.0-h17022bd_0 requires tensorflow 2.5.0, but none of the providers can be installed

I ended up installing with pip.

Best, Marcelo

zoccoler commented 2 years ago

I ended up installing with pip.

An update on this. I had several conflicts of versions, therefore I could not make it work at all. If anyone knows a way to make it work on Windows, I would be grateful! I would really like to try this package in my local machine.

tibuch commented 2 years ago

Which version of Python are you using?

I tried recently Python 3.8 and then conda install tensorflow-gpu, which should install TF 2.4.

But I was working on a Linux machine.

zoccoler commented 2 years ago

I tried with python 3.7 as in the installation instructions.

I just attempted the same commands in a new conda environment with python 3.8, but these package versions do not seem to work there as well :(


  - python 3.8.*

Encountered problems while solving:
  - nothing provides requested tensorflow-gpu 2.4.1**
  - package keras-2.3.1-py37h21ff451_0 requires python >=3.7,<3.8.0a0, but none of the providers can be installed```
cberri commented 2 years ago

Can you post the errors? Which CUDA version are you using? Please also try nvidia-smi and nvcc -V and post the output.

zoccoler commented 2 years ago

Hi @cberri ,

Sorry, I should have posted all versions right from the beginning.

These are the packages in my environment with python 3.8

# packages in environment at C:\Miniconda\envs\n2v-env:
#
# Name                    Version                   Build  Channel
bzip2                     1.0.8                h8ffe710_4    conda-forge
ca-certificates           2021.10.8            h5b45459_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libzlib                   1.2.11            h8ffe710_1014    conda-forge
openssl                   3.0.2                h8ffe710_1    conda-forge
pip                       22.0.4             pyhd8ed1ab_0    conda-forge
python                    3.8.13          hcf16a7b_0_cpython    conda-forge
python_abi                3.8                      2_cp38    conda-forge
setuptools                62.1.0           py38haa244fe_0    conda-forge
sqlite                    3.38.3               h8ffe710_0    conda-forge
tk                        8.6.12               h8ffe710_0    conda-forge
ucrt                      10.0.20348.0         h57928b3_0    conda-forge
vc                        14.2                 hb210afc_6    conda-forge
vs2015_runtime            14.29.30037          h902a5da_6    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xz                        5.2.5                h62dcd97_1    conda-forge

This is my cuda version (11.6):

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Tue_Mar__8_18:36:24_Pacific_Standard_Time_2022
Cuda compilation tools, release 11.6, V11.6.124
Build cuda_11.6.r11.6/compiler.31057947_0
Tue May  3 09:29:22 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 511.65       Driver Version: 511.65       CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ... WDDM  | 00000000:01:00.0 Off |                  N/A |
| N/A   36C    P8     8W /  N/A |      0MiB /  4096MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

I use mamba to install packages. When I try to mamba install tensorflow-gpu I get this error message:

Looking for: ['tensorflow-gpu']

conda-forge/win-64                                          Using cache
conda-forge/noarch                                          Using cache
pkgs/msys2/win-64                                             No change
pkgs/main/win-64                                              No change
pkgs/main/noarch                                              No change
pkgs/r/win-64                                                 No change
pkgs/r/noarch                                                 No change
pkgs/msys2/noarch                                             No change

Pinned packages:
  - python 3.8.*

Encountered problems while solving:
  - package tensorflow-gpu-1.14.0-h0d30ee6_0 requires tensorflow 1.14.0, but none of the providers can be installed

With conda, I get more specific info (attached file)

tensorflow-gpu_conda_error.txt

If you think this is a tensorflow installation problem unrelated to this repo, that's totally fine. Nevertheless, I was wondering if you have a set of package versions (python, tensorflow, etc) tested on Windows 10 that you could recommend. Then I would try to set up my environment aiming for that :)

Best, Marcelo

tibuch commented 2 years ago

Somehow it is trying to install TF 1.14.

Could you try out one of these: https://github.com/CSBDeep/CSBDeep/tree/master/extras#conda-environment

cberri commented 2 years ago

You are using CUDA 11.6. I am not sure if TF can run on such a high CUDA version. At least according to the doc:

https://www.tensorflow.org/install/source_windows

If you scroll down you have a table with the build configurations for Windows 10.

I am running n2v on CUDA 11.0 with TF 2.4.0 installed using pip on Windows 10. Python 3.7.

These are the major steps I followed in case:

  1. Create the n2v conda environment

    $ conda create -n n2v pip python==3.7

  2. Activate the n2v conda environment

    $ conda activate n2v

  3. Install tensorflow and keras

    $ pip install tensorflow-gpu==2.4.0 keras==2.3.1

    Test tensorflow installation

    $ python
$ import tensorflow as tf
$ print("Num GPUs Available: ", len(tf.config.experimental.list_physical_devices('GPU')))

$ Num GPUs Available: X
$ exit()

  4. Install the last n2v release

    $ pip install n2v

zoccoler commented 2 years ago

Ok thank you @tibuch and @cberri ! I will try these options and get back to you as soon as possible.

zoccoler commented 2 years ago

Hi guys,

The installation worked! 🎉 I can confirm from my side that following @cberri instructions works! Thank you!

Let me just report a few extra steps I took before following the instructions:

  1. Downloaded CUDA version 11.0

    nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Thu_Jun_11_22:26:48_Pacific_Daylight_Time_2020
Cuda compilation tools, release 11.0, V11.0.194
Build cuda_11.0_bu.relgpu_drvr445TC445_37.28540450_0

  2. In my case, I noticed I had previous tf and keras versions installed in my base environment (I most likely accidentally installed them there at some point), so I uninstalled Miniconda and also deleted pyhton related folders. Then, reinstalled Miniconda.

  3. Followed @cberri steps up to number 3. The print statement made me aware I had no cuDNN installed (cudnn64_8.dll not found). If I may, this video was what guided me to a proper cuDNN installation.

  4. Finally I installed n2v with pip.

Thanks again guys! I hope this issue may guide Windows 10 users like me to manage the installation.

Best, Marcelo

jdeschamps commented 1 year ago

Newest N2V version bumped the TF dependency version (>2.7), so this issue should now be obsolete.