keisen / tf-keras-vis

Neural network visualization toolkit for tf.keras
https://keisen.github.io/tf-keras-vis-docs/
MIT License
311 stars 45 forks source link

Multi-output will only pass the first output to the score funuction #50

Open frank-qcd-qk opened 3 years ago

frank-qcd-qk commented 3 years ago

https://github.com/keisen/tf-keras-vis/blob/ddd951396f16e7f5b7a0e8619f43f99c599628fb/tf_keras_vis/gradcam.py#L62

Say if I have an output of [tf.tensor , tf.tensor], after this line executed even I use my own scoring function, the obtained "output" is now only the first tf.tensor.

I don't think this is the expected behavior...

Sample model: https://github.com/duckietown/challenge-aido_LF-baseline-behavior-cloning/blob/master/duckieChallenger/frankModel.py

bersbersbers commented 3 years ago

Do you base this on experiments or code analysis? losses should have the same length as self.model.outputs, so this list comprehension should return more than one element:

https://github.com/keisen/tf-keras-vis/blob/ddd951396f16e7f5b7a0e8619f43f99c599628fb/tf_keras_vis/gradcam.py#L51

https://github.com/keisen/tf-keras-vis/blob/b420516ac29ddd9f3f5b964d53a6f3177241d014/tf_keras_vis/__init__.py#L42-L50

Or what am I missing?

frank-qcd-qk commented 3 years ago

I did this base on an experiment...

But maybe I have misunderstood something that could also happen...

I am not sure this losses was working tho... I am using v0.6.0 if that helps.

bersbersbers commented 3 years ago

I did this base on an experiment...

In that case, a short reproducible example would be helpful, I'm pretty sure.

I am using v0.6.0 if that helps.

Note that this has not been released yet as far as I can see.

Also, you may want to look into this code to see what may be going wrong: 9ebe940be6369d0ee40a7a40e8e63e65c3cd888a, although much seems the same except for renamed variables:

https://github.com/keisen/tf-keras-vis/blob/9ebe940be6369d0ee40a7a40e8e63e65c3cd888a/tf_keras_vis/gradcam.py#L69 https://github.com/keisen/tf-keras-vis/blob/9ebe940be6369d0ee40a7a40e8e63e65c3cd888a/tf_keras_vis/gradcam.py#L80 https://github.com/keisen/tf-keras-vis/blob/9ebe940be6369d0ee40a7a40e8e63e65c3cd888a/tf_keras_vis/__init__.py#L42-L50

keisen commented 3 years ago

@frank-qcd-qk , I'm so sorry for late reply. @bersbersbers , Thank you for the detailed explanation!

So Was your problem solved? If not, please let me know that. I will help you resolve it. If resolved, please close this issue.

Thanks!

ck37 commented 1 year ago

Would it be possible to show an example of using tf-keras-vis with a multi-output model? I'm trying to use it for a 3-output model (EfficientNet backbone) but am struggling with how to define the score functions, etc.