How could I apply this KD method to other segmentation teacher/student models?

hunto / DIST_KD

Official implementation of paper "Knowledge Distillation from A Stronger Teacher", NeurIPS 2022

Apache License 2.0

138 stars 20 forks source link

How could I apply this KD method to other segmentation teacher/student models? #5

Closed SophiaMonster closed 1 year ago

SophiaMonster commented 1 year ago

How could I apply this KD method to other teacher/student models and achive the same effect of examples. I tried STDC2 as the teacher model and STDC1 as the student model ,but got bad metric effects, so I can't appreciate more if authors can give some instructions.

hunto commented 1 year ago

Hi @SophiaMonster ,

I am not familiar with STDC and STDC2, but we could try to solve this issue together. I need some detailed information to locate the problem.

What dataset did you use?
Does the DIST loss decrease during training? (https://github.com/hunto/DIST_KD/releases/download/v0.0.1/seg_DIST_deeplabv3_resnet101_resnet18_log.txt for reference.)

hunto commented 1 year ago

I close this issue since no update posted for a long time. If you have any update, please feel free to re-open the issue :)