conda-forge / tensorflow-feedstock

A conda-smithy repository for tensorflow.
BSD 3-Clause "New" or "Revised" License
91 stars 81 forks source link

Tensorflow 2.14 cannot import from keras.legacy_tf_layers #356

Closed hmaarrfk closed 7 months ago

hmaarrfk commented 7 months ago

Solution to issue cannot be found in the documentation.

Issue

mamba create --name tf tensorflow tensorflow-base=2.14=cpu* --channel conda-forge --override-channel
mamba activate tf
python -c "from tensorflow.python.layers import normalization as normalization_layers; normalization_layers.BatchNormalization"
2023-11-28 08:03:35.678566: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: SSE4.1 SSE4.2 AVX AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/mark/miniforge3/envs/tf/lib/python3.11/site-packages/tensorflow/python/layers/normalization.py", line 30, in __getattr__
    return normalization.BatchNormalization
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mark/miniforge3/envs/tf/lib/python3.11/site-packages/tensorflow/python/util/lazy_loader.py", line 66, in __getattr__
    module = self._load()
             ^^^^^^^^^^^^
  File "/home/mark/miniforge3/envs/tf/lib/python3.11/site-packages/tensorflow/python/util/lazy_loader.py", line 49, in _load
    module = importlib.import_module(self.__name__)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mark/miniforge3/envs/tf/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1126, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'keras.legacy_tf_layers'

Installed packages

# packages in environment at /home/mark/miniforge3/envs/tf:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
absl-py                   2.0.0              pyhd8ed1ab_0    conda-forge
aiohttp                   3.9.1           py311h459d7ec_0    conda-forge
aiosignal                 1.3.1              pyhd8ed1ab_0    conda-forge
astunparse                1.6.3              pyhd8ed1ab_0    conda-forge
attrs                     23.1.0             pyh71513ae_1    conda-forge
blinker                   1.7.0              pyhd8ed1ab_0    conda-forge
brotli-python             1.1.0           py311hb755f60_1    conda-forge
bzip2                     1.0.8                hd590300_5    conda-forge
c-ares                    1.22.1               hd590300_0    conda-forge
ca-certificates           2023.11.17           hbcca054_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                5.3.2              pyhd8ed1ab_0    conda-forge
certifi                   2023.11.17         pyhd8ed1ab_0    conda-forge
cffi                      1.16.0          py311hb3a22ac_0    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
click                     8.1.7           unix_pyh707e725_0    conda-forge
cryptography              41.0.7          py311h63ff55d_0    conda-forge
flatbuffers               23.5.26              h59595ed_1    conda-forge
frozenlist                1.4.0           py311h459d7ec_1    conda-forge
gast                      0.5.4              pyhd8ed1ab_0    conda-forge
giflib                    5.2.1                h0b41bf4_3    conda-forge
google-auth               2.23.4             pyhca7485f_0    conda-forge
google-auth-oauthlib      1.0.0              pyhd8ed1ab_1    conda-forge
google-pasta              0.2.0              pyh8c360ce_0    conda-forge
grpcio                    1.54.3          py311hcafe171_0    conda-forge
h5py                      3.10.0          nompi_py311h3839ddf_100    conda-forge
hdf5                      1.14.2          nompi_h4f84152_100    conda-forge
icu                       73.2                 h59595ed_0    conda-forge
idna                      3.6                pyhd8ed1ab_0    conda-forge
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
keras                     2.14.0             pyhd8ed1ab_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.21.2               h659d440_0    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
libabseil                 20230125.3      cxx17_h59595ed_0    conda-forge
libaec                    1.1.2                h59595ed_1    conda-forge
libblas                   3.9.0           20_linux64_openblas    conda-forge
libcblas                  3.9.0           20_linux64_openblas    conda-forge
libcurl                   8.4.0                hca28451_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 13.2.0               h807b86a_3    conda-forge
libgfortran-ng            13.2.0               h69a702a_3    conda-forge
libgfortran5              13.2.0               ha4646dd_3    conda-forge
libgomp                   13.2.0               h807b86a_3    conda-forge
libgrpc                   1.54.3               hb20ce57_0    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
liblapack                 3.9.0           20_linux64_openblas    conda-forge
libnghttp2                1.58.0               h47da74e_0    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libopenblas               0.3.25          pthreads_h413a1c8_0    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
libprotobuf               3.21.12              hfc55251_2    conda-forge
libsqlite                 3.44.2               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_3    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
markdown                  3.5.1              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.3           py311h459d7ec_1    conda-forge
ml_dtypes                 0.2.0           py311h320fe9a_2    conda-forge
multidict                 6.0.4           py311h459d7ec_1    conda-forge
ncurses                   6.4                  h59595ed_2    conda-forge
numpy                     1.26.2          py311h64a7726_0    conda-forge
oauthlib                  3.2.2              pyhd8ed1ab_0    conda-forge
openssl                   3.2.0                hd590300_1    conda-forge
opt_einsum                3.3.0              pyhc1e730c_2    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pip                       23.3.1             pyhd8ed1ab_0    conda-forge
protobuf                  4.21.12         py311hcafe171_0    conda-forge
pyasn1                    0.5.0              pyhd8ed1ab_0    conda-forge
pyasn1-modules            0.3.0              pyhd8ed1ab_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pyjwt                     2.8.0              pyhd8ed1ab_0    conda-forge
pyopenssl                 23.3.0             pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.11.6          hab00c5b_0_cpython    conda-forge
python-flatbuffers        23.5.26            pyhd8ed1ab_0    conda-forge
python_abi                3.11                    4_cp311    conda-forge
pyu2f                     0.1.5              pyhd8ed1ab_0    conda-forge
re2                       2023.03.02           h8c504da_0    conda-forge
readline                  8.2                  h8228510_1    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
requests-oauthlib         1.3.1              pyhd8ed1ab_0    conda-forge
rsa                       4.9                pyhd8ed1ab_0    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.1.10               h9fff704_0    conda-forge
tensorboard               2.14.1             pyhd8ed1ab_0    conda-forge
tensorboard-data-server   0.7.0           py311h63ff55d_1    conda-forge
tensorflow                2.14.0          cpu_py311h4b67847_0    conda-forge
tensorflow-base           2.14.0          cpu_py311ha0be21f_0    conda-forge
tensorflow-estimator      2.14.0          cpu_py311ha26c8b9_0    conda-forge
termcolor                 2.3.0              pyhd8ed1ab_0    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
typing_extensions         4.8.0              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
urllib3                   2.1.0              pyhd8ed1ab_0    conda-forge
werkzeug                  3.0.1              pyhd8ed1ab_0    conda-forge
wheel                     0.42.0             pyhd8ed1ab_0    conda-forge
wrapt                     1.14.1          py311hd4cff14_1    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
yarl                      1.9.3           py311h459d7ec_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               hd590300_5    conda-forge
zstd                      1.5.5                hfc55251_0    conda-forge

Environment info

$ conda info

     active environment : tf
    active env location : /home/mark/miniforge3/envs/tf
            shell level : 2
       user config file : /home/mark/.condarc
 populated config files : /home/mark/miniforge3/.condarc
                          /home/mark/.condarc
          conda version : 23.10.0
    conda-build version : 3.27.0
         python version : 3.10.13.final.0
       virtual packages : __archspec=1=zen
                          __cuda=12.2=0
                          __glibc=2.35=0
                          __linux=5.19.0=0
                          __unix=0=0
       base environment : /home/mark/miniforge3  (writable)
      conda av data dir : /home/mark/miniforge3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /home/mark/miniforge3/pkgs
                          /home/mark/.conda/pkgs
       envs directories : /home/mark/miniforge3/envs
                          /home/mark/.conda/envs
               platform : linux-64
             user-agent : conda/23.10.0 requests/2.31.0 CPython/3.10.13 Linux/5.19.0-50-generic ubuntu/22.04.3 glibc/2.35 solver/libmamba conda-libmamba-solver/23.11.1 libmambapy/1.5.3
                UID:GID : 1003:1003
             netrc file : None
           offline mode : False
hmaarrfk commented 7 months ago

For what its worth, the same thing happens on tensorflow 2.15 installed from pip

mamba create --name tf python=3.11
mamba activate tf
pip install tensorflow tf_keras
python -c "from tensorflow.python.layers import normalization as normalization_layers; normalization_layers.BatchNormalization"
2023-11-28 08:08:47.469714: I external/local_tsl/tsl/cuda/cudart_stub.cc:31] Could not find cuda drivers on your machine, GPU will not be used.
2023-11-28 08:08:47.512597: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2023-11-28 08:08:47.512642: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2023-11-28 08:08:47.513608: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2023-11-28 08:08:47.519596: I external/local_tsl/tsl/cuda/cudart_stub.cc:31] Could not find cuda drivers on your machine, GPU will not be used.
2023-11-28 08:08:47.519785: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-11-28 08:08:48.298875: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/mark/miniforge3/envs/tf/lib/python3.11/site-packages/tensorflow/python/layers/normalization.py", line 30, in __getattr__
    return normalization.BatchNormalization
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mark/miniforge3/envs/tf/lib/python3.11/site-packages/tensorflow/python/util/lazy_loader.py", line 67, in __getattr__
    module = self._load()
             ^^^^^^^^^^^^
  File "/home/mark/miniforge3/envs/tf/lib/python3.11/site-packages/tensorflow/python/util/lazy_loader.py", line 50, in _load
    module = importlib.import_module(self.__name__)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mark/miniforge3/envs/tf/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1126, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'tf_keras.legacy_tf_layers'
hmaarrfk commented 7 months ago

I believe its just an internal API layer that is still an appendage.

tf-slim should release a new update to incorporate the fixes they have already done: https://github.com/google-research/tf-slim/issues/24

they have ignored my patch for more than 1 year https://github.com/google-research/tf-slim/pull/21

so i'm going to see what other maintainers think of https://github.com/conda-forge/tf-slim-feedstock/pull/6