What is the difference between distilled model and fine-tuned model?

SwinTransformer / Feature-Distillation

MIT License

235 stars 11 forks source link

What is the difference between distilled model and fine-tuned model? #15

Closed shenyehui closed 1 year ago

shenyehui commented 1 year ago

Is it the teacher model that becomes the distilled model after feature distillation, and then the distilled model becomes the fine-tuning model after fine-tuning?