About model architecture

[jcjlin]

Thank you for sharing this paper, it's very insightful! I noticed in the paper that you used RD as the baseline, and MemRD achieved 99.6%, but I'm a bit confused about the model part. It seems that both the teacher and student use the same CNN (encoder) architecture in figure3, rather than the encoder-decoder architecture used in RD?

[SimonThomine]

Hello, To clarify, I'm not the author of the paper so the code is based on my understanding. I believe that the paper introduce the memory part as a tool for knowledge distillation for AD and explains the method for the simple case of knowledge distillation (forward distillation). However, in the implementation details, they said "We choose RD as the baseline model". For me, this means that the architecture is adapted within the reverse distillation paradigm. If your understanding of the paper (and supp materials) is different from mine, please let me know and we can discuss it.

[jcjlin]

Hi, Thanks for the reply. I'm just a bit confused about the name about this paper since in abstract it mentioned MemKD while in abaltion study it was called MemRD. Based on my understanding, in RD method we input image only to the teacher network but not both Student Teacher network. Please let me know if there's any misunderstandings.

[SimonThomine]

You're right ! In the code I have implemented the two versions, one with the classic forward distillation and the other one with reverse distillation. If you have any questions regarding the implementation, let me know !

[jcjlin]

May I ask how did you implement RD version in your code? Is it slightly different from the method mentioned in the paper?

[SimonThomine]

I used the implementation from the reverse distillation paper (RD4AD github) and I modified it slightly to add the memory part.