search

dvlab-research / ReviewKD

Distilling Knowledge via Knowledge Review, CVPR 2021
255 stars 34 forks source link

preact setting #11

Open fingerk28 opened 3 years ago

fingerk28 commented 3 years ago

Line 222, ReviewKD/CIFAR-100/train.py t_features, t_pred = teacher(images, is_feat = True, preact=True)

Line 76, ReviewKD/CIFAR-100/model/reviewkd.py student_features = self.student(x,is_feat=True)

Why preact is set different value in these two positions? One is True, the other is False (default value) thanks!

akuxcw commented 3 years ago

Thank you for this question! Actually, we didn't pay much attention to this part. I'm not sure whether the performance will be higher if we align these positions. You are welcome to try it by yourself!

Mdahao commented 3 years ago

Thank you for this question! Actually, we didn't pay much attention to this part. I'm not sure whether the performance will be higher if we align these positions. You are welcome to try it by yourself!

Hi sir, student_features = self.student(x,is_feat=True)..... What is the role of 'is_feat' ？，I am a little puzzled.

fingerk28 commented 3 years ago

Thank you for this question! Actually, we didn't pay much attention to this part. I'm not sure whether the performance will be higher if we align these positions. You are welcome to try it by yourself!

Hi sir, student_features = self.student(x,is_feat=True)..... What is the role of 'is_feat' ？，I am a little puzzled.

If is_feat=True, model will return logits and feature map, otherwise just return logits.

Mdahao commented 3 years ago

Thank you for this question! Actually, we didn't pay much attention to this part. I'm not sure whether the performance will be higher if we align these positions. You are welcome to try it by yourself!

Hi sir, student_features = self.student(x,is_feat=True)..... What is the role of 'is_feat' ？，I am a little puzzled.

If is_feat=True, model will return logits and feature map, otherwise just return logits.

Thank you for this reply, sir, and one question, is that the location called "review."？ 82 for features, abf, shape, out_shape in zip(x[1:], self.abfs[1:], self.shapes[1:], self.out_shapes[1:]):