search

1202kbs / Understanding-NN

Tensorflow tutorial for various Deep Neural Network visualization techniques
MIT License
345 stars 90 forks source link

Contribution Help #2

Closed zengzhi2015 closed 6 years ago

zengzhi2015 commented 6 years ago

Hi, friend. My name is Zhi Zeng working in Xidian University in China. I really appreciate your project. Last week, I found a new brilliant work proposed by Google Brain, called "Learning how to explain neural networks: PatternNet and PatternAttribution". The result seems much better than its ancestors. So, I would love to realize the code (not released along side with the paper) and contribute this to your project. However, I am so new to this kind of project and to black-box interpretations. Would you love to share me with some advises? My email address is zhizeng@xidian.edu.cn.

1202kbs commented 6 years ago

@zengzhi2015

Hi, nice to meet you!

I'm directly commenting here because the email you gave me does not seem to exist.

Thank you for appreciating my work :) I'm glad you found it useful. I agree that this repo is slightly outdated so I'll be happy if you could implement the more recent work(s) and contribute to this repo.

I also happened to be reading the PatternNet PatternAttribution paper by Google Brain, but I haven't fully understood the contents yet so I may not be able to help you out with specific details. Maybe you could create a new branch and we could work on it together when I finish reading the paper.

As for the advises, here are some things I found useful when implementing explainability works:

  1. Before coding, make sure you fully understand the proposed algorithm. Otherwise, you will be easily confused and end up wasting a lot of time debugging trivial details.

  2. Make sure you understand how the tensor shapes change at each step of the algorithm. Having in your head all the tensor shape changes will prevent confusion.

  3. I recommend first implementing the algorithm on small toy networks (e.g. a 3-layer fully connected network trained on MNIST) to get the hang of it. Then you can move onto larger networks.

Good luck!

zengzhi2015 commented 6 years ago

@1202kbs Hi, friend! Thanks a lot for so much valuable advises! I am sorry that there is a little typo-error in my previous email address. (zhizeng@mail.xidian.edu.cn; zengzhi_2012@hotmail.com). I would love to follow your suggestions. I will contact you when there is a progress.