Closed skyqwe123 closed 1 year ago
@huawei-noah-admin @ggjy Thanks for your project. When I Training on ImageNet-1K
To train a DeiT-Tiny student with a Cait-S24 teacher, run: python -m torch.distributed.launch --nproc_per_node=8 main.py --distributed --model deit_tiny_patch16_224 --teacher-model cait_s24_224 --distillation-type soft --distillation-alpha 1 --distillation-beta 1 --w-sample 0.1 --w-patch 4 --w-rand 0.2 --K 192 --s-id 0 1 2 3 8 9 10 11 --t-id 0 1 2 3 20 21 22 23 --drop-path 0
customized_forward.py
def vit_forward_features(self, x, require_feat: bool = False): x = self.patch_embed(x) cls_token = self.cls_token.expand(x.shape[0], -1, -1) if self.dist_token is None: x = torch.cat((cls_token, x), dim=1)
{AttributeError}'VisionTransformer' object has no attribute 'dist_token'
@huawei-noah-admin @ggjy Thanks for your project. When I Training on ImageNet-1K
To train a DeiT-Tiny student with a Cait-S24 teacher, run: python -m torch.distributed.launch --nproc_per_node=8 main.py --distributed --model deit_tiny_patch16_224 --teacher-model cait_s24_224 --distillation-type soft --distillation-alpha 1 --distillation-beta 1 --w-sample 0.1 --w-patch 4 --w-rand 0.2 --K 192 --s-id 0 1 2 3 8 9 10 11 --t-id 0 1 2 3 20 21 22 23 --drop-path 0
customized_forward.py
def vit_forward_features(self, x, require_feat: bool = False): x = self.patch_embed(x) cls_token = self.cls_token.expand(x.shape[0], -1, -1) if self.dist_token is None: x = torch.cat((cls_token, x), dim=1)
{AttributeError}'VisionTransformer' object has no attribute 'dist_token'