search

FlyEgle / MAE-pytorch

Masked Autoencoders Are Scalable Vision Learners
245 stars 36 forks source link

Training time question #2

Closed yanxiangyi closed 2 years ago

yanxiangyi commented 2 years ago

Thanks for the quick implementaion! How long do you think it will take if I train it on 8 x 2080Ti?

FlyEgle commented 2 years ago

I am so sad about, if you follow the paper training strategy, may not have enough time to training 1600 epoch with 4k batchsize. Because it apply on TPU. So, if you want see the pretrain results, you can training vit-tiny 100 epoch with 64 or 32 bs for one 2080ti, may have a coarse result. And if you want to finetune vit-tiny, you can load the pretrain models and follow the deit training strategy. At least one week you can get the finish result.