albermax / innvestigate

A toolbox to iNNvestigate neural networks' predictions!
Other
1.24k stars 235 forks source link

[Question] Innvestigate brakes tensorflow <-> keras compatibility #305

Closed fornasierov closed 1 year ago

fornasierov commented 1 year ago

Description

I have a simple conda environment in a Kaggle notebook. This is the conda list | grep 'tensor\|keras' output:

keras                     2.6.0                    pypi_0    pypi
keras-preprocessing       1.1.2                    pypi_0    pypi
keras-tuner               1.1.2                    pypi_0    pypi
tensorboard               2.6.0                    pypi_0    pypi
tensorboard-data-server   0.6.1                    pypi_0    pypi
tensorboard-plugin-wit    1.8.1                    pypi_0    pypi
tensorboardx              2.5.1                    pypi_0    pypi
tensorflow                2.6.4                    pypi_0    pypi
tensorflow-addons         0.14.0                   pypi_0    pypi
tensorflow-cloud          0.1.14                   pypi_0    pypi
tensorflow-datasets       4.3.0                    pypi_0    pypi
tensorflow-decision-forests 0.2.0                    pypi_0    pypi
tensorflow-estimator      2.6.0                    pypi_0    pypi
tensorflow-gcs-config     2.6.0                    pypi_0    pypi
tensorflow-hub            0.12.0                   pypi_0    pypi
tensorflow-io             0.21.0                   pypi_0    pypi
tensorflow-metadata       1.9.0                    pypi_0    pypi
tensorflow-probability    0.14.1                   pypi_0    pypi
tensorflow-serving-api    2.9.0                    pypi_0    pypi
tensorflow-transform      1.9.0                    pypi_0    pypi
tensorpack                0.11                     pypi_0    pypi
tensorstore               0.1.28                   pypi_0    pypi

When I pip install innvestigate it downgrades my current Keras version (2.6.0). See:

Collecting innvestigate
  Downloading innvestigate-1.0.9-py3-none-any.whl (100 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.6/100.6 kB 603.8 kB/s eta 0:00:00a 0:00:01
Requirement already satisfied: pytest in /opt/conda/lib/python3.7/site-packages (from innvestigate) (7.2.0)
Requirement already satisfied: scipy in /opt/conda/lib/python3.7/site-packages (from innvestigate) (1.7.3)
Requirement already satisfied: pillow in /opt/conda/lib/python3.7/site-packages (from innvestigate) (9.1.1)
Collecting keras==2.2.4
  Downloading Keras-2.2.4-py2.py3-none-any.whl (312 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 312.5/312.5 kB 1.9 MB/s eta 0:00:00a 0:00:01
Requirement already satisfied: future in /opt/conda/lib/python3.7/site-packages (from innvestigate) (0.18.2)
Requirement already satisfied: h5py in /opt/conda/lib/python3.7/site-packages (from innvestigate) (3.7.0)
Requirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from innvestigate) (1.21.6)
Requirement already satisfied: six>=1.9.0 in /opt/conda/lib/python3.7/site-packages (from keras==2.2.4->innvestigate) (1.15.0)
Requirement already satisfied: pyyaml in /opt/conda/lib/python3.7/site-packages (from keras==2.2.4->innvestigate) (6.0)
Requirement already satisfied: keras-preprocessing>=1.0.5 in /opt/conda/lib/python3.7/site-packages (from keras==2.2.4->innvestigate) (1.1.2)
Collecting keras-applications>=1.0.6
  Downloading Keras_Applications-1.0.8-py3-none-any.whl (50 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.7/50.7 kB 4.3 MB/s eta 0:00:00
Requirement already satisfied: attrs>=19.2.0 in /opt/conda/lib/python3.7/site-packages (from pytest->innvestigate) (21.4.0)
Requirement already satisfied: packaging in /opt/conda/lib/python3.7/site-packages (from pytest->innvestigate) (21.3)
Requirement already satisfied: tomli>=1.0.0 in /opt/conda/lib/python3.7/site-packages (from pytest->innvestigate) (2.0.1)
Requirement already satisfied: exceptiongroup>=1.0.0rc8 in /opt/conda/lib/python3.7/site-packages (from pytest->innvestigate) (1.0.4)
Requirement already satisfied: pluggy<2.0,>=0.12 in /opt/conda/lib/python3.7/site-packages (from pytest->innvestigate) (1.0.0)
Requirement already satisfied: iniconfig in /opt/conda/lib/python3.7/site-packages (from pytest->innvestigate) (1.1.1)
Requirement already satisfied: importlib-metadata>=0.12 in /opt/conda/lib/python3.7/site-packages (from pytest->innvestigate) (4.13.0)
Requirement already satisfied: zipp>=0.5 in /opt/conda/lib/python3.7/site-packages (from importlib-metadata>=0.12->pytest->innvestigate) (3.8.0)
Requirement already satisfied: typing-extensions>=3.6.4 in /opt/conda/lib/python3.7/site-packages (from importlib-metadata>=0.12->pytest->innvestigate) (4.1.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging->pytest->innvestigate) (3.0.9)
Installing collected packages: keras-applications, keras, innvestigate
  Attempting uninstall: keras
    Found existing installation: keras 2.6.0
    Uninstalling keras-2.6.0:
      Successfully uninstalled keras-2.6.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow-io 0.21.0 requires tensorflow-io-gcs-filesystem==0.21.0, which is not installed.
tensorflow 2.6.4 requires h5py~=3.1.0, but you have h5py 3.7.0 which is incompatible.
tensorflow 2.6.4 requires keras<2.7,>=2.6.0, but you have keras 2.2.4 which is incompatible.
tensorflow 2.6.4 requires numpy~=1.19.2, but you have numpy 1.21.6 which is incompatible.
tensorflow 2.6.4 requires typing-extensions<3.11,>=3.7, but you have typing-extensions 4.1.1 which is incompatible.
tensorflow-transform 1.9.0 requires tensorflow!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,<2.10,>=1.15.5, but you have tensorflow 2.6.4 which is incompatible.
tensorflow-serving-api 2.9.0 requires tensorflow<3,>=2.9.0, but you have tensorflow 2.6.4 which is incompatible.
Successfully installed innvestigate-1.0.9 keras-2.2.4 keras-applications-1.0.8
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: [https://pip.pypa.io/warnings/venv](https://pip.pypa.io/warnings/venv%3C/span%3E%3Cspan)[ class="ansi-yellow-fg">](https://pip.pypa.io/warnings/venv%3C/span%3E%3Cspan)

After that, if I repeat conda list | grep 'tensor\|keras' I get:

keras                     2.2.4                    pypi_0    pypi
keras-applications        1.0.8                    pypi_0    pypi
keras-preprocessing       1.1.2                    pypi_0    pypi
keras-tuner               1.1.2                    pypi_0    pypi
tensorboard               2.6.0                    pypi_0    pypi
tensorboard-data-server   0.6.1                    pypi_0    pypi
tensorboard-plugin-wit    1.8.1                    pypi_0    pypi
tensorboardx              2.5.1                    pypi_0    pypi
tensorflow                2.6.4                    pypi_0    pypi
tensorflow-addons         0.14.0                   pypi_0    pypi
tensorflow-cloud          0.1.14                   pypi_0    pypi
tensorflow-datasets       4.3.0                    pypi_0    pypi
tensorflow-decision-forests 0.2.0                    pypi_0    pypi
tensorflow-estimator      2.6.0                    pypi_0    pypi
tensorflow-gcs-config     2.6.0                    pypi_0    pypi
tensorflow-hub            0.12.0                   pypi_0    pypi
tensorflow-io             0.21.0                   pypi_0    pypi
tensorflow-metadata       1.9.0                    pypi_0    pypi
tensorflow-probability    0.14.1                   pypi_0    pypi
tensorflow-serving-api    2.9.0                    pypi_0    pypi
tensorflow-transform      1.9.0                    pypi_0    pypi
tensorpack                0.11                     pypi_0    pypi
tensorstore               0.1.28                   pypi_0    pypi

As you can see, I have now Keras 2.2.4 together with TensorFlow 2.6.4.

The model I'm working on uses a specific loss function which can be found in the tensorflow_addons.losses module. But after installing innvestigate in my environment, I cannot import tensorflow_addons. See the output of import tensorflow_addons as tfa:

Using TensorFlow backend.
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/tmp/ipykernel_24/802459132.py in <module>
----> 1 import tensorflow_addons as tfa

/opt/conda/lib/python3.7/site-packages/tensorflow_addons/__init__.py in <module>
     19 
     20 # Local project imports
---> 21 from tensorflow_addons import activations
     22 from tensorflow_addons import callbacks
     23 from tensorflow_addons import image

/opt/conda/lib/python3.7/site-packages/tensorflow_addons/activations/__init__.py in <module>
     15 """Additional activation functions."""
     16 
---> 17 from tensorflow_addons.activations.gelu import gelu
     18 from tensorflow_addons.activations.hardshrink import hardshrink
     19 from tensorflow_addons.activations.lisht import lisht

/opt/conda/lib/python3.7/site-packages/tensorflow_addons/activations/gelu.py in <module>
     17 import warnings
     18 
---> 19 from tensorflow_addons.utils.types import TensorLike
     20 
     21 

/opt/conda/lib/python3.7/site-packages/tensorflow_addons/utils/types.py in <module>
     22 # TODO: Remove once https://github.com/tensorflow/tensorflow/issues/44613 is resolved
     23 if tf.__version__[:3] > "2.5":
---> 24     from keras.engine import keras_tensor
     25 else:
     26     from tensorflow.python.keras.engine import keras_tensor

ImportError: cannot import name 'keras_tensor' from 'keras.engine' (/opt/conda/lib/python3.7/site-packages/keras/engine/__init__.py)

If I do not install innvestigate and thus do not downgrade Keras from version 2.6.0 to 2.2.4 I'm able to import tensorflow_addons without a problem.

As can be seen in the README from the official tensorflow_addons repository, tensorflow_addons 0.14.0 support TF 2.4, 2.5, 2.6, and I have TF 2.6 installed, so it should work, but because there is a dependency between Keras and Tensorflow, it is not possible to import keras_tensor from keras.engine in Keras version 2.2.4. For instance, if I run from keras.engine import keras_tensor in Keras 2.6.0 I get no error.

Takeaway

I am opening this as a question because I believe it is necessary to signal to users that downgrading Keras without properly downgrading TF to its compatible version will break some TF model implementations.

I'm hoping to get your opinions on that.

Platform information

adrhill commented 1 year ago

Hi @fornasierov, looking at your platform information, you appear to be using iNNvestigate v1.0.9, a two-year-old release which was based on TF1. This old version of iNNvestigate forces pip to install an old version of Keras.

Updating to iNNvestigate 2, which is based on TF2, should fix your issue. The latest release is v2.0.1.

fornasierov commented 1 year ago

Aditional Information

When I use pip index versions innvestigate to show all available versions inside of a Kaggle notebook I get the following:

WARNING: pip index is currently an experimental command. It may be removed/changed in a future release without prior warning.
innvestigate (1.0.9)
Available versions: 1.0.9, 1.0.8
Note: you may need to restart the kernel to use updated packages.

Upon seeing this, I decided to check in my local machine (Pop!_OS 22.04 LTS) and I was able to see the newest versions:

WARNING: pip index is currently an experimental command. It may be removed/changed in a future release without prior warning.
innvestigate (2.0.1)
Available versions: 2.0.1, 2.0.0, 1.0.9, 1.0.8

It was all due to Python 3.7. Although I was using TF 2, the Python version was incompatible with innvestigate 2. See the output of pip install innvestigate==2.0.1:

ERROR: Ignored the following versions that require a different python version: 2.0.0 Requires-Python >=3.8,<3.11; 2.0.1 Requires-Python >=3.8,<3.11
ERROR: Could not find a version that satisfies the requirement innvestigate==2.0.1 (from versions: 1.0.8, 1.0.9)
ERROR: No matching distribution found for innvestigate==2.0.1

This issue is closed and I was able to make it work. Thank you all!

adrhill commented 1 year ago

Thanks for the follow-up, this is very helpful for other people on Python 3.7.