AnacondaRecipes / tensorflow_recipes

Tensorflow conda recipes
27 stars 26 forks source link

Tensorflow 2.5 under Conda forces a requirement on incompatible Numpy 1.20 #32

Closed torzdf closed 1 year ago

torzdf commented 3 years ago

This is a duplicate post from: https://github.com/ContinuumIO/anaconda-issues/issues/12604

I apologize if this is bad practice. I just want to make sure that this bug gets visibility, as it is a major blocker for our app and RTX30xx users.

Actual Behavior

The Anaconda package for Tensorflow 2.5 forces a requirement on Numpy>=1.20. This version of Numpy is fundamentally incompatible with Tensorflow 2.5 and makes it unusable for many training functions.

It is then not possible to downgrade the numpy version as Conda is unable to resolve the dependencies, leading to a massive list of incompatible packages.

See here for reference: https://github.com/tensorflow/tensorflow/issues/50204 https://stackoverflow.com/questions/58479556/notimplementederror-cannot-convert-a-symbolic-tensor-2nd-target0-to-a-numpy

Expected Behavior

Tensorflow should pin numpy at <1.20

Steps to Reproduce

Set up environment, activate and enter interpreter

Under Windows:

conda create -n tensorflow_test python=3.8 tensorflow-gpu=2.5
conda activate tensorflow_test
python

Build a toy model for testing:

>>> import tensorflow as tf
>>> model = tf.keras.Sequential()
>>> model.add(tf.keras.layers.LSTM(50, activation='relu', input_shape=(28, 28)))

Output:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\training\tracking\base.py", line 522, in _method_wrapper
    result = method(self, *args, **kwargs)
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\keras\engine\sequential.py", line 213, in add
    layer(x)
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\keras\layers\recurrent.py", line 668, in __call__
    return super(RNN, self).__call__(inputs, **kwargs)
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 969, in __call__
    return self._functional_construction_call(inputs, args, kwargs,
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 1107, in _functional_construction_call
    outputs = self._keras_tensor_symbolic_call(
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 840, in _keras_tensor_symbolic_call
    return self._infer_output_signature(inputs, args, kwargs, input_masks)
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 880, in _infer_output_signature
    outputs = call_fn(inputs, *args, **kwargs)
    inputs, initial_state, _ = self._process_inputs(inputs, initial_state, None)
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\keras\layers\recurrent.py", line 868, in _process_inputs
    initial_state = self.get_initial_state(inputs)
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\keras\layers\recurrent.py", line 650, in get_initial_state
    init_state = get_initial_state_fn(
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\keras\layers\recurrent.py", line 2516, in get_initial_state
    return list(_generate_zero_filled_state_for_cell(
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\keras\layers\recurrent.py", line 2998, in _generate_zero_filled_state_for_cell
    return _generate_zero_filled_state(batch_size, cell.state_size, dtype)
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\keras\layers\recurrent.py", line 3014, in _generate_zero_filled_state
    return nest.map_structure(create_zeros, state_size)
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\util\nest.py", line 867, in map_structure
    structure[0], [func(*x) for x in entries],
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\util\nest.py", line 867, in <listcomp>
    structure[0], [func(*x) for x in entries],
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\keras\layers\recurrent.py", line 3011, in create_zeros
    return array_ops.zeros(init_state_size, dtype=dtype)
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\util\dispatch.py", line 206, in wrapper
    return target(*args, **kwargs)
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\ops\array_ops.py", line 2911, in wrapped
    tensor = fun(*args, **kwargs)
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\ops\array_ops.py", line 2960, in zeros
    output = _constant_if_small(zero, shape, dtype, name)
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\ops\array_ops.py", line 2896, in _constant_if_small
    if np.prod(shape) < 1000:
  File "<__array_function__ internals>", line 5, in prod
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\numpy\core\fromnumeric.py", line 3030, in prod
    return _wrapreduction(a, np.multiply, 'prod', axis, dtype, out,
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\numpy\core\fromnumeric.py", line 87, in _wrapreduction
    return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
  File "D:\miniconda3\envs\tensorflow_test\lib\site-packages\tensorflow\python\framework\ops.py", line 867, in __array__
    raise NotImplementedError(
NotImplementedError: Cannot convert a symbolic Tensor (lstm/strided_slice:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported

Attempt to downgrade numpy:

conda install numpy"<1.20"

I will spare you the full output of conflicts here as it is pages and pages.

Anaconda or Miniconda version:
https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe
Operating System:

Windows 10

conda info
     active environment : tensorflow_test
    active env location : D:\miniconda3\envs\tensorflow_test
            shell level : 3
       user config file : C:\Users\xxxx\.condarc
 populated config files : C:\Users\xxxx\.condarc
          conda version : 4.10.3
    conda-build version : not installed
         python version : 3.8.5.final.0
       virtual packages : __cuda=11.3=0
                          __win=0=0
                          __archspec=1=x86_64
       base environment : D:\miniconda3  (writable)
      conda av data dir : D:\miniconda3\etc\conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : D:\miniconda3\pkgs
                          C:\Users\xxxx\.conda\pkgs
                          C:\Users\xxxx\AppData\Local\conda\conda\pkgs
       envs directories : D:\miniconda3\envs
                          C:\Users\xxxx\.conda\envs
                          C:\Users\xxxx\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/4.10.3 requests/2.25.1 CPython/3.8.5 Windows/10 Windows/10.0.19041
          administrator : False
             netrc file : None
           offline mode : False
conda list --show-channel-urls
# packages in environment at D:\miniconda3\envs\tensorflow_test:
#
# Name                    Version                   Build  Channel
_tflow_select             2.1.0                       gpu    defaults
abseil-cpp                20210324.2           hd77b12b_0    defaults
absl-py                   0.13.0           py38haa95532_0    defaults
aiohttp                   3.7.4            py38h2bbff1b_1    defaults
astor                     0.8.1            py38haa95532_0    defaults
astunparse                1.6.3                      py_0    defaults
async-timeout             3.0.1            py38haa95532_0    defaults
attrs                     21.2.0             pyhd3eb1b0_0    defaults
blas                      1.0                         mkl    defaults
blinker                   1.4              py38haa95532_0    defaults
brotlipy                  0.7.0           py38h2bbff1b_1003    defaults
ca-certificates           2021.7.5             haa95532_1    defaults
cached-property           1.5.2                      py_0    defaults
cachetools                4.2.2              pyhd3eb1b0_0    defaults
certifi                   2021.5.30        py38haa95532_0    defaults
cffi                      1.14.6           py38h2bbff1b_0    defaults
chardet                   3.0.4           py38haa95532_1003    defaults
charset-normalizer        2.0.4              pyhd3eb1b0_0    defaults
click                     8.0.1              pyhd3eb1b0_0    defaults
coverage                  5.5              py38h2bbff1b_2    defaults
cryptography              3.4.7            py38h71e12ea_0    defaults
cudatoolkit               11.3.1               h59b6b97_2    defaults
cudnn                     8.2.1                cuda11.3_0    defaults
cython                    0.29.24          py38hd77b12b_0    defaults
flatbuffers               2.0.0                h6c2663c_0    defaults
gast                      0.4.0              pyhd3eb1b0_0    defaults
giflib                    5.2.1                h62dcd97_0    defaults
google-auth               1.33.0             pyhd3eb1b0_0    defaults
google-auth-oauthlib      0.4.1                      py_2    defaults
google-pasta              0.2.0              pyhd3eb1b0_0    defaults
grpcio                    1.36.1           py38hc60d5dd_1    defaults
h5py                      3.2.1            py38h3de5c98_0    defaults
hdf5                      1.10.6               h7ebc959_0    defaults
icc_rt                    2019.0.0             h0cc432a_1    defaults
icu                       68.1                 h6c2663c_0    defaults
idna                      3.2                pyhd3eb1b0_0    defaults
importlib-metadata        3.10.0           py38haa95532_0    defaults
intel-openmp              2021.3.0          haa95532_3372    defaults
jpeg                      9b                   hb83a4c4_2    defaults
keras-preprocessing       1.1.2              pyhd3eb1b0_0    defaults
libcurl                   7.78.0               h86230a5_0    defaults
libpng                    1.6.37               h2a8f88b_0    defaults
libprotobuf               3.14.0               h23ce68f_0    defaults
libssh2                   1.9.0                h7a1dbc1_1    defaults
markdown                  3.3.4            py38haa95532_0    defaults
mkl                       2021.3.0           haa95532_524    defaults
mkl-service               2.4.0            py38h2bbff1b_0    defaults
mkl_fft                   1.3.0            py38h277e83a_2    defaults
mkl_random                1.2.2            py38hf11a4ad_0    defaults
multidict                 5.1.0            py38h2bbff1b_2    defaults
numpy                     1.20.3           py38ha4e8547_0    defaults
numpy-base                1.20.3           py38hc2deb75_0    defaults
oauthlib                  3.1.1              pyhd3eb1b0_0    defaults
openssl                   1.1.1l               h2bbff1b_0    defaults
opt_einsum                3.3.0              pyhd3eb1b0_1    defaults
pip                       21.2.2           py38haa95532_0    defaults
protobuf                  3.14.0           py38hd77b12b_1    defaults
pyasn1                    0.4.8              pyhd3eb1b0_0    defaults
pyasn1-modules            0.2.8                      py_0    defaults
pycparser                 2.20                       py_2    defaults
pyjwt                     2.1.0            py38haa95532_0    defaults
pyopenssl                 20.0.1             pyhd3eb1b0_1    defaults
pyreadline                2.1                      py38_1    defaults
pysocks                   1.7.1            py38haa95532_0    defaults
python                    3.8.11               h6244533_1    defaults
python-flatbuffers        1.12               pyhd3eb1b0_0    defaults
requests                  2.26.0             pyhd3eb1b0_0    defaults
requests-oauthlib         1.3.0                      py_0    defaults
rsa                       4.7.2              pyhd3eb1b0_1    defaults
scipy                     1.7.1            py38hbe87c03_2    defaults
setuptools                52.0.0           py38haa95532_0    defaults
six                       1.16.0             pyhd3eb1b0_0    defaults
snappy                    1.1.8                h33f27b4_0    defaults
sqlite                    3.36.0               h2bbff1b_0    defaults
tensorboard               2.5.0                      py_0    defaults
tensorboard-plugin-wit    1.6.0                      py_0    defaults
tensorflow                2.5.0           gpu_py38h8e8c102_0    defaults
tensorflow-base           2.5.0           gpu_py38hb3da07e_0    defaults
tensorflow-estimator      2.5.0              pyh7b7c402_0    defaults
tensorflow-gpu            2.5.0                h17022bd_0    defaults
termcolor                 1.1.0            py38haa95532_1    defaults
typing-extensions         3.10.0.0             hd3eb1b0_0    defaults
typing_extensions         3.10.0.0           pyh06a4308_0    defaults
urllib3                   1.26.6             pyhd3eb1b0_1    defaults
vc                        14.2                 h21ff451_1    defaults
vs2015_runtime            14.27.29016          h5e58377_2    defaults
werkzeug                  1.0.1              pyhd3eb1b0_0    defaults
wheel                     0.35.1             pyhd3eb1b0_0    defaults
win_inet_pton             1.1.0            py38haa95532_0    defaults
wincertstore              0.2                      py38_0    defaults
wrapt                     1.12.1           py38he774522_1    defaults
yarl                      1.6.3            py38h2bbff1b_0    defaults
zipp                      3.5.0              pyhd3eb1b0_0    defaults
zlib                      1.2.11               h62dcd97_4    defaults