Closed szubing closed 2 years ago
Hi @szubing, thanks for your interest!
The two models use the same architecture but were trained differently. DeiT uses knowledge distillation while ViT uses plain supervision for training. I recommend reading the papers to understand the differences.
I have not found any difference between vit_base_patch16_224 and deit_base_patch16_224 in ViT_LRP.py? What makes they differerent?