search

aliyun / Revisiting-Knowledge-Distillation-an-Inheritance-and-Exploration-Framework

13 stars 4 forks source link

Found an error in processor/IE-KD.py #2

Open Park-ing-lot opened 3 years ago

Park-ing-lot commented 3 years ago

Thank you for providing your code. It is helpful for studying Knowledge distillation on Images.

But, while I try to check the performance IE-KD using this repository, There is a code that I don't understand well, so I leave a question.

In the paper and the model architecture, It seems the student's feature map is divided into two parts, then they are doing inheritance and exploration respectively.

However, in your code of this repository, the teacher's feature map is split not the student's.

I want to ask if the code misrepresents the teacher and student, whether it is a simple error, or what is the right way.

guanqj932 commented 2 years ago

I also found this minor issue. The F_S and f_s are also not present in the figure.

dineshdaultani commented 2 years ago

Hi @CallessCaller, I am also trying to reproduce the paper for a while now. And also agree with you about the conflict between paper and repo implementation of student and teacher network. I think most of the code is such that the teacher is treated as a student and vice versa.

There's a lot of ambiguity in the code due to the above issue. I wonder if maybe the paper writers forked their original code but made a slight bug about the above issue in the code. Let me know what do you think? Thanks!