My generated image for "integrated_gradients.py" was totally black.

[r0cketr1kky]

I know this isn't the right way to talk about this problem but I didn't know how to contact the maintainers. I don't know why this happens. I tried it out for different images, but to no avail. I also tried changing the parameter n_steps yet I got a black image.

[matheushent]

It happened to me and I fixed changing transform_to_normalized_grayscale function.

change: grayscale_tensor = tf.reduce_sum(tensor, axis=-1)

to: grayscale_tensor = tf.math.reduce_sum(tensor, axis=-1, keepdims=True)

[r0cketr1kky]

Hi, thanks for your reply @matheushent . Still no luck. I changed it and I still get the black image...

[matheushent]

Hi, thanks for your reply @matheushent . Still no luck. I changed it and I still get the black image...

Try to plot _grayscale_integratedgradients inside explain function. If it works so you found the problem is in grid_display function. Remember _grayscale_integratedgradients is a 4D array (batch_size, height, width, channels).

[r0cketr1kky]

Will it work if I remove that function? Because I don’t see any point of it, other transforming it into grayscale.

On Mon, 6 Apr 2020 at 4:55 PM, Matheus Tosta notifications@github.com wrote:

Hi, thanks for your reply @matheushent https://github.com/matheushent . Still no luck. I changed it and I still get the black image...

Try to plot grayscale_integrated_gradients inside explain function. If it works so you found the problem is in grid_display function. Remember grayscale_integrated_gradients is a 4D array (batch_size, height, width, channels).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/sicara/tf-explain/issues/125#issuecomment-609735432, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJLQTTZMOXI6L7QZH57DTTTRLG3Y3ANCNFSM4MA6ZMAQ .

[matheushent]

Will it work if I remove that function? Because I don’t see any point of it, other transforming it into grayscale. … On Mon, 6 Apr 2020 at 4:55 PM, Matheus Tosta @.**> wrote: Hi, thanks for your reply @matheushent https://github.com/matheushent . Still no luck. I changed it and I still get the black image... Try to plot grayscale_integrated_gradients inside explain function. If it works so you found the problem is in grid_display function. Remember grayscale_integrated_gradients* is a 4D array (batch_size, height, width, channels). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#125 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJLQTTZMOXI6L7QZH57DTTTRLG3Y3ANCNFSM4MA6ZMAQ .

Indeed I don't use that function, so it will. I just return _grayscale_integratedgradients and work with it knowing it is a 4D array. Furthermore, doing that, you can get all pictures without using the grid, so you can work separately on each image.

[r0cketr1kky]

Do you have any example code for me to look at for working with a single image? I’m still having trouble, even after removing the grayscale function.

On Mon, 6 Apr 2020 at 6:16 PM, Matheus Tosta notifications@github.com wrote:

Will it work if I remove that function? Because I don’t see any point of it, other transforming it into grayscale. … <#m-4162699721386692276> On Mon, 6 Apr 2020 at 4:55 PM, Matheus Tosta @.**> wrote: Hi, thanks for your reply @matheushent https://github.com/matheushent https://github.com/matheushent . Still no luck. I changed it and I still get the black image... Try to plot grayscale_integrated_gradients inside explain function. If it works so you found the problem is in grid_display function. Remember grayscale_integrated_gradients* is a 4D array (batch_size, height, width, channels). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#125 (comment) https://github.com/sicara/tf-explain/issues/125#issuecomment-609735432>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJLQTTZMOXI6L7QZH57DTTTRLG3Y3ANCNFSM4MA6ZMAQ .

Indeed I don't use that function, so it will. I just return grayscale_integrated_gradients and work with it knowing it is a 4D array. Furthermore, doing that, you can get all pictures without using the grid, so I can work separately on each image.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/sicara/tf-explain/issues/125#issuecomment-609772099, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJLQTT542GOUJXZRHFSCI4DRLHFJFANCNFSM4MA6ZMAQ .

[RaphaelMeudec]

Hi everyone! The problem comes from the facts that th enumber of channels in the map can be > 3, which is unhandy to plot. Reducing to 1 channel only aims at representing it more easily.

In the next tf-explain release, I'll make sure to split the generation of the 4D tensor from the generation of the visualization, so that anyone can use attribution map the way they want.

[r0cketr1kky]

When I use the transform_to_normalized_grayscale function, I get a tensor with all values 0(with shape (1,400,400,1)), but when I just return integrated_gradients in the get_integrated_gradients function, I get the gradients(with shape (1,400,400,3)). I think there's something wrong while changing my image to grayscale. Is there any alternate way to reduce the no of channels?

[rao208]

Hi, thanks for your reply @matheushent . Still no luck. I changed it and I still get the black image...

Try to plot _grayscale_integratedgradients inside explain function. If it works so you found the problem is in grid_display function. Remember _grayscale_integratedgradients is a 4D array (batch_size, height, width, channels).

@matheushent What is the use of grid_display function? I went through the code and I get the use of everything except for the grid_display function. I was wondering what is the significance of that function? What if we do not use that function?

[matheushent]

Hi, thanks for your reply @matheushent . Still no luck. I changed it and I still get the black image...

Try to plot _grayscale_integratedgradients inside explain function. If it works so you found the problem is in grid_display function. Remember _grayscale_integratedgradients is a 4D array (batch_size, height, width, channels).

@matheushent What is the use of grid_display function? I went through the code and I get the use of everything except for the grid_display function. I was wondering what is the significance of that function? What if we do not use that function?

In my understanding, _griddisplay function only concatenates all images into one. When disabling it you need to change the way things happen on callback. For example, here is the example of what I did in the core:

def explain(self, validation_data, model, class_index, n_steps=10, _grid=True):
        """
        Compute Integrated Gradients for a specific class index

        Args:
            validation_data (Tuple[np.ndarray, Optional[np.ndarray]]): Validation data
                to perform the method on. Tuple containing (x, y).
            model (tf.keras.Model): tf.keras model to inspect
            class_index (int): Index of targeted class
            n_steps (int): Number of steps in the path
            _grid (bool): Whether display images on grid or separately.

        Returns:
            np.ndarray: Grid of all the integrated gradients or 4D array (batch_size, height, width, channels)
        """
        images, _ = validation_data

        interpolated_images = IntegratedGradients.generate_interpolations(
            images, n_steps
        )

        integrated_gradients = IntegratedGradients.get_integrated_gradients(
            interpolated_images, model, class_index, n_steps
        )

        grayscale_integrated_gradients = transform_to_normalized_grayscale(
            tf.abs(integrated_gradients)
        ).numpy()

        if _grid:
            grid = grid_display(grayscale_integrated_gradients)
            return grid
        else:
            return grayscale_integrated_gradients

Note that doing it and setting _grid as false, _grayscale_integratedgradients will return a 4D array of shape (batch_size, H, W, N), so a error will be raised since here it is:

with self.file_writer.as_default():
    tf.summary.image(
        "IntegratedGradients", np.expand_dims([grid], axis=-1), step=epoch
    )

Note np.expand_dims([grid], axis=-1) will be a 5D array. So, since you're not using _griddisplay, you'll need to write:

def on_epoch_end(self, epoch, logs=None):
        """
        Draw Integrated Gradients outputs at each epoch end to Tensorboard.

        Args:
            epoch (int): Epoch index
            logs (dict): Additional information on epoch
        """
        explainer = IntegratedGradients()
        images = explainer.explain(
            self.validation_data, self.model, self.class_index, self.n_steps
        )

        # Using the file writer, log the reshaped image.
        with self.file_writer.as_default():
            tf.summary.image(
                "IntegratedGradients", images, step=epoch
            )

Also, I recommend you to change max_outputs parameter of tf.summary.image according your needs since the default is 3.

[rao208]

@matheushent So, basically grid_display is required only

1. if we use callbacks instead of core i.e. IntegratedGradientsCallback(Callback) instead of IntegratedGradients

grid_display function only concatenates all images into one

2. if we give a list of images test_data = ([img1, img2....etc], None)

Is that correct?

[matheushent]

@matheushent So, basically grid_display is required only

1. if we use callbacks instead of core i.e. IntegratedGradientsCallback(Callback) instead of IntegratedGradients

grid_display function only concatenates all images into one

1. if we give a list of images test_data = ([img1, img2....etc], None)

Is that correct?

@rao208 Yes, but note you need to pass a list [img1, img2, etc...] using specifically integrated gradients since here the code stacks the images. Using GradCAM you'll need to stack by yourself for example.

[rao208]

@matheushent Oh okay... now it is clear...Thank you :)

[VictorW96]

This also happened to me for other Gradient based Saliency Map methods. For my purpose i could fix it by returning the gradients directly without applying the transform_to_normalized_grayscale function