TypeError: unhashable type: 'numpy.ndarray'

[aymeric75]

Hello,

Thanks for your work, I am trying to use it on the VAE of latplan , I tried this:

attributions = de.explain("guidedbp", latent, latent, train_subdata_batch_cache)

the 2nd argument should be a _conceptscore but I don't understand what you mean by that exactly (Latplan's encoder, encodes an image into a latent vector... that's it ), could you explain it to me please ?

With the code above I get the following error: TypeError: unhashable type: 'numpy.ndarray'

latent and _train_subdata_batchcache are both numpy arrays, both with the following shape (400, 2, 28, 28, 3) where 400 is the batch size, 2 is because there are two image per example (it's a "sequence" of two images) and 28x28x3 is the image dimension.

Thanks a lot!

Aymeric

[lenbrocki]

Hi,

the concept score is a similarity measure between the concept vector and the latent representation of the sample that you want to obtain the saliency map for. It is therefore a scalar, i.e. just a single number, rather than an array.

The procedure of obtaining a concept saliency map is as follows: 1) Obtain a concept vector, by choosing samples that represent the concept you are interested in and obtain their latent vectors and average them 2) Calculate the concept score, e.g. dot product, between the concept vector and latent representation of your images 3) Feed the concept score into the de.explain function to obtain the concept salieny maps, ideally it will highlight which parts of the image drive the similarity between the concept and the samples in img_array

latent_vector = encoder(input_tensor)
concept_score = K.sum(latent_vector*concept_vector) # dot product
attributions = de.explain(method, concept_score, input_tensor, img_array)

[aymeric75]

Hi,

the concept score is a similarity measure between the concept vector and the latent representation of the sample that you want to obtain the saliency map for. It is therefore a scalar, i.e. just a single number, rather than an array.

The procedure of obtaining a concept saliency map is as follows:

1. Obtain a concept vector, by choosing samples that represent the concept you are interested in and obtain their latent vectors and average them

2. Calculate the concept score, e.g. dot product, between the concept vector and latent representation of your images

3. Feed the concept score into the de.explain function to obtain the concept salieny maps, ideally it will highlight which parts of the image drive the similarity between the concept and the samples in `img_array`

latent_vector = encoder(input_tensor)
concept_score = K.sum(latent_vector*concept_vector) # dot product
attributions = de.explain(method, concept_score, input_tensor, img_array)

Thanks but in Latplan we do not have a concept vector. What I would like is simply to see which parts of the image is/are responsible for activating a specific neuron of the latent vector.

If my latent vector is [0 1 0 0 1] and I want to see what part of the image correspond to the 2nde neuron, what should I do ? Maybe specifying a "concept vector" equal to [0 1 0 0 0] so that [0 1 0 0 1] * [0 1 0 0 0] = [0 1 0 0 0] ? I am not sure...

[aymeric75]

Here is my new code without the unhashable error:

attributions = de.explain("guidedbp", 1, tf.convert_to_tensor(latent[0,0,:,:,:]), train_subdata_batch_cache[0,0,:,:,:]) But now I get :

TypeError: Fetch argument None has invalid type <class 'NoneType'> Which is similar to the other post but my code is alreay using Tensorflow, so I don't know

p.s: my work should end up with a research paper, on which you will be cited of course, if I use your method... (tell me if you want more infos)

[lenbrocki]

If you want so see what image parts correspond to a single dimension of a given latent vector you can indeed just replace concept_score with that dimension, e.g. latent[0,0,:,:,:]. The function call should then be something like this

attributions = de.explain("guidedbp", latent[0,0,:,:,:], input_tensor, sample_img)

This should create a saliency map with respect to the dimension 0 of the latent vector of sample_img[0] Also, this repository has been created 4 years ago so I can't guarantee that the code still works with recent versions of tensorflow/keras.

[aymeric75]

I think there is a confusion here, actually latent[0,0,:,:,:] IS the latent vector (the first two 0s are just for taking the 1st element of the batch and the first vector from the two images - because images are fed two by two to the network).

Regarding the error that I get, here is the complete log:

  File latplan/network.py line 605 function train : attributions = de.explain("guidedbp", 1, tf.convert_to_tensor(latent[0,0,:,:,:]), train_subdata_batch_cache[0,0,:,:,:])
                        val_data = ['<numpy.ndarray float64  (1000, 2, 28, 28, 3)>']
                     val_data_to = ['<numpy.ndarray float64  (1000, 2, 28, 28, 3)>']
                          resume = True
                          kwargs = {'test_noise': False, 'test_hard': True, 'train_noise': True, 'train_hard': False, 'dropout_z': False, 'noise': 0.2, 'dropout': 0.2, 'optimizer': 'radam', 'min_temperature': 0.5, 'epoch': 2000, 'gs_annealing_start': 0, 'gs_annealing_end': 1000, 'kl_cycle_start': 2000, 'clipnorm': 0.1, 'batch_size': 400, 'lr': 0.001, 'N': 300, 'zerosuppress': 0.1, 'densify': False, 'max_temperature': 5.0, 'conv_channel': 32, 'conv_channel_increment': 1, 'conv_kernel': 5, 'conv_pooling': 1, 'conv_per_pooling': 1, 'conv_depth': 3, 'fc_width': 100, 'fc_depth': 2, 'A': 6000, 'aae_activation': 'relu', 'aae_width': 1000, 'aae_depth': 2, 'eff_regularizer': None, 'beta_d': 1000, 'beta_z': 10, 'output': 'GaussianOutput(sigma=0.1)', 'mode': 'learn', 'track': 'sokoban_image-20000-global-global-2-train', 'num_examples': 20000, 'aeclass': 'CubeSpaceAE_AMA4Conv', 'comment': 'kltune2', 'generator': 'latplan.puzzle.sokoban', 'picsize': [[28, 28, 3]], 'mean': [[[0.858823529411715, 0.83
                              de = <deepexplain.tensorflow.methods.DeepExplain object at 0x7ff3c49b4a30>
                           epoch = 0
                     start_epoch = 0
                     index_array = '<numpy.ndarray int64    (17999,)>'
                           clist = <keras.callbacks.CallbackList object at 0x7ff3b54aee80>
                     transitions = '<numpy.ndarray float64  (19999, 2, 28, 28, 3)>'
                               _ = '<numpy.ndarray float64  (39998, 28, 28, 3)>'
                            logs = {}
                     batch_count = 0
                   indices_cache = ['<numpy.ndarray int64    (400,)>', '<numpy.ndarray int64    (400,)>', '<numpy.ndarray int64    (400,)>', '...<41 more>']
                train_data_cache = [['<numpy.ndarray float64  (400, 2, 28, 28, 3)>'], ['<numpy.ndarray float64  (400, 2, 28, 28, 3)>'], ['<numpy.ndarray float64  (400, 2, 28, 28, 3)>'], '...<41 more>']
             train_data_to_cache = [['<numpy.ndarray float64  (400, 2, 28, 28, 3)>'], ['<numpy.ndarray float64  (400, 2, 28, 28, 3)>'], ['<numpy.ndarray float64  (400, 2, 28, 28, 3)>'], '...<41 more>']
                     problem_dir = 'samples/sokoban_sokoban_image-20000-global-global-2-train_20000_CubeSpaceAE_AMA4Conv_kltune2/logs/05-11T07:43:01.194_2500_With/sokoban-2-False/003-000'
                      domainfile = 'samples/sokoban_sokoban_image-20000-global-global-2-train_20000_CubeSpaceAE_AMA4Conv_kltune2/logs/05-11T07:43:01.194_2500_With/domain.pddl'
                       tracefile = 'samples/sokoban_sokoban_image-20000-global-global-2-train_20000_CubeSpaceAE_AMA4Conv_kltune2/logs/05-11T07:43:01.194_2500_With/sokoban-2-False/003-000/tracefile.trace'
                         csvfile = 'samples/sokoban_sokoban_image-20000-global-global-2-train_20000_CubeSpaceAE_AMA4Conv_kltune2/logs/05-11T07:43:01.194_2500_With/sokoban-2-False/003-000/plan.csv'
                         pngfile = 'samples/sokoban_sokoban_image-20000-global-global-2-train_20000_CubeSpaceAE_AMA4Conv_kltune2/logs/05-11T07:43:01.194_2500_With/sokoban-2-False/003-000/problem.png'
                        jsonfile = 'samples/sokoban_sokoban_image-20000-global-global-2-train_20000_CubeSpaceAE_AMA4Conv_kltune2/logs/05-11T07:43:01.194_2500_With/sokoban-2-False/003-000/problem.json'
                         logfile = 'samples/sokoban_sokoban_image-20000-global-global-2-train_20000_CubeSpaceAE_AMA4Conv_kltune2/logs/05-11T07:43:01.194_2500_With/sokoban-2-False/003-000/problem.log'
                         npzfile = 'samples/sokoban_sokoban_image-20000-global-global-2-train_20000_CubeSpaceAE_AMA4Conv_kltune2/logs/05-11T07:43:01.194_2500_With/sokoban-2-False/003-000/problem.npz'
                         negfile = 'samples/sokoban_sokoban_image-20000-global-global-2-train_20000_CubeSpaceAE_AMA4Conv_kltune2/logs/05-11T07:43:01.194_2500_With/sokoban-2-False/003-000/problem.negative'
                        planfile = 'samples/sokoban_sokoban_image-20000-global-global-2-train_20000_CubeSpaceAE_AMA4Conv_kltune2/logs/05-11T07:43:01.194_2500_With/sokoban-2-False/003-000/file.plan'
                         sasfile = 'samples/sokoban_sokoban_image-20000-global-global-2-train_20000_CubeSpaceAE_AMA4Conv_kltune2/logs/05-11T07:43:01.194_2500_With/sokoban-2-False/003-000/sasfile.sas'
             train_subdata_cache = ['<numpy.ndarray float64  (400, 2, 28, 28, 3)>']
          train_subdata_to_cache = ['<numpy.ndarray float64  (400, 2, 28, 28, 3)>']
                             net = <keras.engine.training.Model object at 0x7ff3b7b06280>
       train_subdata_batch_cache = '<numpy.ndarray float64  (400, 2, 28, 28, 3)>'
    train_subdata_to_batch_cache = '<numpy.ndarray float64  (400, 2, 28, 28, 3)>'
                          latent = '<numpy.ndarray float32  (400, 2, 28, 28, 3)>'
                      batch_size = 400
                        clipnorm = 0.1
                              lr = 0.001
                       optimizer = 'radam'
                   plot_val_data = '<numpy.ndarray float64  (1, 2, 28, 28, 3)>'
                            self = <latplan.model.ConvolutionalConcreteDetNormalizedLogitAddBidirectionalTransitionAEPlus object at 0x7ff264e8adc0>
                      train_data = ['<numpy.ndarray float64  (17999, 2, 28, 28, 3)>']
                   train_data_to = ['<numpy.ndarray float64  (17999, 2, 28, 28, 3)>']

  File deepexplain/tensorflow/methods.py line 735 function explain : result = method.run()
                    self = <deepexplain.tensorflow.methods.DeepExplain object at 0x7ff3c49b4a30>
                  method = <deepexplain.tensorflow.methods.GuidedBackpropagation object at 0x7ff3b5428610>
                       T = 1
                       X = <tf.Tensor 'Const_14:0' shape=(28, 28, 3) dtype=float32>
                      xs = '<numpy.ndarray float64  (28, 28, 3)>'
                  kwargs = {}
             method_flag = 7

  File deepexplain/tensorflow/methods.py line 130 function run : results =  self.session_run(attributions, self.xs)
                    self = <deepexplain.tensorflow.methods.GuidedBackpropagation object at 0x7ff3b5428610>
            attributions = [None]

  File deepexplain/tensorflow/methods.py line 96 function session_run : return self.session.run(T, feed_dict)
                    self = <deepexplain.tensorflow.methods.GuidedBackpropagation object at 0x7ff3b5428610>
                       T = [None]
                      xs = '<numpy.ndarray float64  (28, 28, 3)>'
               feed_dict = {<tf.Tensor 'Const_14:0' shape=(28, 28, 3) dtype=float32>: '<numpy.ndarray float64  (28, 28, 3)>', <tf.Tensor 'keras_learning_phase/input:0' shape=() dtype=bool>: 0}

  File ../../usr/local/lib/python3.8/dist-packages/tensorflow_core/python/client/session.py line 955 function run : result = self._run(None, fetches, feed_dict, options_ptr,
                    self = <tensorflow.python.client.session.Session object at 0x7ff199f9ac40>
                 fetches = [None]
               feed_dict = {<tf.Tensor 'Const_14:0' shape=(28, 28, 3) dtype=float32>: '<numpy.ndarray float64  (28, 28, 3)>', <tf.Tensor 'keras_learning_phase/input:0' shape=() dtype=bool>: 0}
                 options = None
            run_metadata = None
             options_ptr = None
        run_metadata_ptr = None

  File ../../usr/local/lib/python3.8/dist-packages/tensorflow_core/python/client/session.py line 1164 function _run : fetch_handler = _FetchHandler(
                     self = <tensorflow.python.client.session.Session object at 0x7ff199f9ac40>
                   handle = None
                  fetches = [None]
                feed_dict = {<tf.Tensor 'Const_14:0' shape=(28, 28, 3) dtype=float32>: '<numpy.ndarray float64  (28, 28, 3)>', <tf.Tensor 'keras_learning_phase/input:0' shape=() dtype=bool>: 0}
                  options = None
             run_metadata = None
         feed_dict_tensor = ObjectIdentityDictionary({<_ObjectIdentityWrapper wrapping <tf.Tensor 'Const_14:0' shape=(28, 28, 3) dtype=float32>>: array([[[ 0.9999998 ,  1.        ,  1.        ],
        [ 0.9999998 ,  1.        , -0.99999994],
        [ 0.9999998 ,  1.        ,  1.        ],
        ...,
        [ 0.9999998 ,  1.        , -0.99999994],
        [ 0.9999998 ,  1.        ,  1.        ],
        [ 1.        ,  1.        ,  1.        ]],

       [[ 0.9999998 ,  1.        ,  1.        ],
        [ 0.9999998 ,  1.        ,  1.        ],
        [ 0.9999998 ,  1.        ,  1.        ],
        ...,
        [ 0.9999998 ,  1.        ,  1.        ],
        [ 0.9999998 ,  1.        ,  1.        ],
        [ 0.9999998 ,  1.        ,  1.        ]],

       [[-0.99999994, -0.9999999 , -0.99999994],
        [ 0.9999998 ,  1.        ,  1.        ],
        [ 0.9999998 ,  1.        ,  1.        ],
        ...,
        [ 0.9999998 ,  1.        ,  1.        ],
        [ 0.9999998 ,  1.  
                 feed_map = {b'Const_14:0': (<tf.Tensor 'Const_14:0' shape=(28, 28, 3) dtype=float32>, '<numpy.ndarray float64  (28, 28, 3)>'), b'keras_learning_phase/input:0': (<tf.Tensor 'keras_learning_phase/input:0' shape=() dtype=bool>, 0)}
             feed_handles = {}
                     feed = <tf.Tensor 'keras_learning_phase/input:0' shape=() dtype=bool>
                 feed_val = 0
                  subfeed = <tf.Tensor 'keras_learning_phase/input:0' shape=() dtype=bool>
              subfeed_val = 0
                subfeed_t = <tf.Tensor 'keras_learning_phase/input:0' shape=() dtype=bool>
    is_tensor_handle_feed = False
                   np_val = '<numpy.ndarray bool     ()>'

  File ../../usr/local/lib/python3.8/dist-packages/tensorflow_core/python/client/session.py line 474 function __init__ : self._fetch_mapper = _FetchMapper.for_fetch(fetches)
                    self = <tensorflow.python.client.session._FetchHandler object at 0x7ff3b5428970>
                   graph = <tensorflow.python.framework.ops.Graph object at 0x7ff199fa67f0>
                 fetches = [None]
            feed_handles = {}
                   feeds = ObjectIdentityDictionary({<_ObjectIdentityWrapper wrapping <tf.Tensor 'Const_14:0' shape=(28, 28, 3) dtype=float32>>: array([[[ 0.9999998 ,  1.        ,  1.        ],
        [ 0.9999998 ,  1.        , -0.99999994],
        [ 0.9999998 ,  1.        ,  1.        ],
        ...,
        [ 0.9999998 ,  1.        , -0.99999994],
        [ 0.9999998 ,  1.        ,  1.        ],
        [ 1.        ,  1.        ,  1.        ]],

       [[ 0.9999998 ,  1.        ,  1.        ],
        [ 0.9999998 ,  1.        ,  1.        ],
        [ 0.9999998 ,  1.        ,  1.        ],
        ...,
        [ 0.9999998 ,  1.        ,  1.        ],
        [ 0.9999998 ,  1.        ,  1.        ],
        [ 0.9999998 ,  1.        ,  1.        ]],

       [[-0.99999994, -0.9999999 , -0.99999994],
        [ 0.9999998 ,  1.        ,  1.        ],
        [ 0.9999998 ,  1.        ,  1.        ],
        ...,
        [ 0.9999998 ,  1.        ,  1.        ],
        [ 0.9999998 ,  1.   

  File ../../usr/local/lib/python3.8/dist-packages/tensorflow_core/python/client/session.py line 266 function for_fetch : return _ListFetchMapper(fetch)
                   fetch = [None]

  File ../../usr/local/lib/python3.8/dist-packages/tensorflow_core/python/client/session.py line 375 function __init__ : self._mappers = [_FetchMapper.for_fetch(fetch) for fetch in fetches]
                    self = <tensorflow.python.client.session._ListFetchMapper object at 0x7ff3b545a100>
                 fetches = [None]

  File ../../usr/local/lib/python3.8/dist-packages/tensorflow_core/python/client/session.py line 375 function <listcomp> : self._mappers = [_FetchMapper.for_fetch(fetch) for fetch in fetches]
                      .0 = <list_iterator object at 0x7ff3b545a220>
                   fetch = None

  File ../../usr/local/lib/python3.8/dist-packages/tensorflow_core/python/client/session.py line 262 function for_fetch : raise TypeError('Fetch argument %r has invalid type %r' %
                   fetch = None

TypeError: Fetch argument None has invalid type <class 'NoneType'>

[lenbrocki]

I see, in that case you can input instead of concept_score for example latent[0,0,0,:,:] or any other dimension of the latent representation. I think the error could arise because you don't put in the correct input_tensor in attributions = de.explain(method, concept_score, input_tensor, img_array). It should be something like input_tensor = keras.Input(shape=img_shape). Please refer to the notebooks for details

[aymeric75]

in the README you mention the jupyter notebook to be in the paper, but it's not....

input_tensor = keras.Input(shape=img_shape) would be an empty tensor... are you sure about this ?

Could you just give an example of the type (tensor, np array or whatnot) and the shape of the arguments of explain

attributions = de.explain(method, concept_score, input_tensor, img_array)

concept_score: we said it should be a number, ok, I tried : 1 input_tensor: this should be the latent vector right ? img_array: this should be the tensor representing the input image ?

Thanks

[lenbrocki]

I have been referring to this notebook. Probably the easiest will be to take this as a starting point and replace the VAE with the model you are working with. I'm not sure if TensorFlow/Keras still works like that, I have switched to PyTorch since then, but at the time of writing the code the logic was as follow. Consider the example from the linked notebook

input_img = keras.Input(shape=img_shape) 

x = layers.Conv2D(64, 5, padding='same', activation='relu', strides=(2,2))(input_img)
x = layers.BatchNormalization()(x)

[...]

with DeepExplain(session=sess, graph=sess.graph) as de:

    input_tensor = input_img
    method = 'guidedbp'

    latents = encoder(input_tensor)

    concept_score = [K.sum(latents*i) for i in concept_vectors[attr]]
    attributions_guided = [de.explain(method, i, input_tensor, img_array) for i in concept_score]

input_tensor is the input layer to your tensorflow model. Passing it to the de.explain function will result in feeding img_array to the encoder and you obtain latents

concept_score: this should be a scalar quantity derived from the latent representation of a sample image. If you are interested only in a single dimension it should be for exampleconcept_score = latents[0,0] assuming that latents has dimensions (batch_size, latent_dim) and you want to visualize the zero-th dimension of the latent representation of the zero-th image.

I hope this is clearing things up a bit.

[aymeric75]

thx