search

yitu-opensource / T2T-ViT

ICCV2021, Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet
Other
1.14k stars 177 forks source link

gained 0.094 for eval_top1, and 0.33 for eval_top5, after 36-epoch training on 8 gpus #56

Open CheerM opened 3 years ago

CheerM commented 3 years ago

hi, would you mind releasing the training log for T2t-vit-t-14 training with 8 GPUs? I tried to rerun the script for training T2t-vit-t-14 with 8 GPUs. It gained 0.094 for eval_top1, 0.33 for eval_top5, after 36 epochs. It seems too slow to converge.

lucastononrodrigues commented 3 years ago

I don't know about the convergence over time but it should take 310 epochs to get the paper results.

yuanli2333 commented 3 years ago

hi, would you mind releasing the training log for T2t-vit-t-14 training with 8 GPUs? I tried to rerun the script for training T2t-vit-t-14 with 8 GPUs. It gained 0.094 for eval_top1, 0.33 for eval_top5, after 36 epochs. It seems too slow to converge.

Hi, the log of T2t-vit-t-14 is trained with 8 GPUs. It's normal if your results are slightly higher or lower than the logs.

WangChen0902 commented 2 years ago

hi, would you mind releasing the training log for T2t-vit-t-14 training with 8 GPUs? I tried to rerun the script for training T2t-vit-t-14 with 8 GPUs. It gained 0.094 for eval_top1, 0.33 for eval_top5, after 36 epochs. It seems too slow to converge.

Hello, have you solved the problem? I have the same problem. And the loss doesn't decrease.

imkzh commented 2 years ago

Same here, training T2t-vit-t-14 on 3 GPU with -b64, and after 80 epochs: top-1 acc = 0.095%, top-5 acc = 0.301%

It seems no improvements from epoch 20 (Top1=0.093, Top5=0.3199) through 80.