search

marcoancona / DeepExplain

A unified framework of perturbation and gradient-based attribution methods for Deep Neural Networks interpretability. DeepExplain also includes support for Shapley Values sampling. (ICLR 2018)
https://arxiv.org/abs/1711.06104
MIT License
720 stars 133 forks source link

Difficulties when trying to import DeepExplain to Google Colab #59

Open SebFAU opened 4 years ago

SebFAU commented 4 years ago

Expected behaviour

When using the command:

!pip install -e git+https://github.com/marcoancona/DeepExplain.git#egg=deepexplain

in the interactive session of Google colab, the installation succeeds and you should be able to import the library via:

from deepexplain.tensorflow import DeepExplain

Actual behaviour

The import fails with a "Module not found" error. That's why I tried the following workaround after solving the environment on Google's Linux-Server:

import sys; sys.path.append("/content/src/deepexplain"); from deepexplain.tensorflow import DeepExplain

which results in a Key Error.

Backtrace

`KeyError Traceback (most recent call last)

in () 3 import sys 4 sys.path.append("/content/src/deepexplain") ----> 5 from deepexplain.tensorflow import DeepExplain 3 frames /content/src/deepexplain/deepexplain/tensorflow/__init__.py in () ----> 1 from .methods import DeepExplain /content/src/deepexplain/deepexplain/tensorflow/methods.py in () 544 545 --> 546 @ops.RegisterGradient("DeepExplainGrad") 547 def deepexplain_grad(op, grad): 548 global _ENABLED_METHOD_CLASS, _GRAD_OVERRIDE_CHECKFLAG /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in __call__(self, f) 2438 def __call__(self, f): 2439 """Registers the function `f` as gradient function for `op_type`.""" -> 2440 _gradient_registry.register(f, self._op_type) 2441 return f 2442 /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/registry.py in register(self, candidate, name) 59 "Registering two %s with name '%s'! " 60 "(Previous registration was in %s %s:%d)" % ---> 61 (self._name, name, frame.name, frame.filename, frame.lineno)) 62 63 logging.vlog(1, "Registering %s (%s) in %s.", name, candidate, self._name) KeyError: "Registering two gradient with name 'DeepExplainGrad'! (Previous registration was in register /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/registry.py:66)"` Do you have any suggestion on how to best use your library in colab ? Maybe the solution is straight forward and I'm just struggling to see it.
flol commented 4 years ago

I had the same problem and found that restarting the runtime after installing DeepExplain usually helps.

So after connecting to a runtime the first cell I run just installs DeepExplain:

import sys
!{sys.executable} -m pip install -e git+https://github.com/marcoancona/DeepExplain.git#egg=deepexplain

Then restart the runtime via Runtime -> restart runtime. After the runtime comes back just continue with importing DeepExplain. If you run into the error module 'tensorflow' has no attribute 'get_default_session' make sure to force loading Tensorflow 1.x, since Google Colab now uses Tensorflow 2.x by default. So my second cell in the notebook (and the first I run after restarting the runtime) is

%tensorflow_version 1.x
from deepexplain.tensorflow import DeepExplain
timonmerk commented 3 years ago

I have the same error in jupyter notebook. Might be an issue with tensorflow 2