[Any Hints?] - TF-explain for YOLOv3

[Meywether]

Hello everyone :)

Does anyone has experience by using tf-explain or any other explainability lib for using it with YOLOv3?

I am looking pretty long for it, but only found this one: feature vis yolo v3 https://github.com/jennalau/feature-vis-yolov3

But it is not working, because a lot of files are missing and the author is not responding ...

Many thanks in advance!

Meywether

[johnny-mueller]

@Meywether Which implementation of yolov3 do you use and which error message do you currently receive? I am also working on this question and maybe we can benefit from each other.

[Meywether]

Hello @johnny-mueller, i tried it with yolov3 TF1 and TF2 but I never got it running. I always fail because I can not get beyond passing the class index to the lib. I am currently trying to access the layers shape in a keras tf 2 implementation. Next step would be to access the heatmap for all classes. But I am not so far now. ;)

[johnny-mueller]

@Meywether If I understand you correctly, you are currently trying to create the model using keras.model(input_layer, output_layer)? if this is the case, then we are currently in the same spot. Unfortunately, I can't really get any further, but maybe you have an approach at the moment?

[Meywether]

@johnny-mueller Yes, we are at the same spot. The main problem, out of my perspective is, that we are loosing the connection between the class props after the nms of yolo, before the yolo head. I will update this issue if I come any further !

[johnny-mueller]

@Meywether I am also of the opinion that it is a connection problem. I permanently get a message that he cannot find input_07. But if you take a closer look at the model you see that there is an input_06 layer and only then an input_09 again. I can't find input_07 and input08 myself. If I should come to a result, I would also contact you.

[Meywether]

@johnny-mueller Maybe I can help you: I found out that (my Yolo) consists out of multiple submodels (= Keras.Model instance). The input errors occurs also at my code.

Maybe this information can help you more than me XD

[johnny-mueller]

@Meywether I think I could solve the problem. I have another problem a few lines below but the input should work. I don't know the implementation you use but it was enough for me to add the input layers manually. Here is an example that works for me:

inputs= [model.layers[0].input] + [model.layers[2].layers[0].input] + [model.layers[6].layers[0].input]
last_conv_layer = model.get_layer('yolo_output_1')
last_conv_layer_model = keras.model(inputs, last_conv_layer.output)

But my next problem that I could not solve so far lies in this line. So if anyone has a suggestion, I would be happy to hear about it.

with tf.GradientTape() as tape:
    # Compute activations of the last conv layer and make the tape watch it
    last_conv_layer_output = last_conv_layer_model(img_array)

The error message that appears is: "AssertionError: Could not compute output Tensor("lambda_1/Identity:0", shape=(None, None, None, 3, 85), dtype=float32)"

[Meywether]

Congrats! @johnny-mueller Do you have a github repo where you can add me ? Maybe I can help you/ you me ^^ It seems to be a TF 2 implementation ? Shouldn't it be keras.Model instead of keras.model?

[johnny-mueller]

@Meywether Currently I have no repo but I use this repo for the Yolov3 with TF2: https://github.com/zzh8829/yolov3-tf2

And yes, it must be "keras.Model()"

[Meywether]

Fine! same starting repo! But does it work with keras.Model instead of your keras.mode ?

[johnny-mueller]

@Meywether No it does not. It was just a copy&paste error. In my code it was right

[Meywether]

The error message that appears is: "AssertionError: Could not compute output Tensor("lambda_1/Identity:0", shape=(None, None, None, 3, 85), dtype=float32)"

perfect. mmh whats your input shape of the image? or the layer before?

[johnny-mueller]

Input processing is the same as in the detect.py file from the original yolov3 repo

[Meywether]

mmh ok, can you try to get the output shape of the previous layer before the error occurs?

[johnny-mueller]

shape = (None, None, None, 256)

[johnny-mueller]

@Meywether here the file is uploaded. the code is still very unorganized and chaotic.

https://www.file-upload.net/download-14080821/grad-cam.py.html

[Meywether]

shape = (None, None, None, 256)

(None, None, None, 3, 85) -> from your code above, it seems that, whyever, there is a shape issue, need to check you code, when i am back at my pc :)

[johnny-mueller]

@Meywether any suggestions to solve the current issue?

[Meywether]

@Meywether any suggestions to solve the current issue?

No, I did not come any further sorry -.- Need to improve my understanding of these things in more detail ...