search

facebookresearch / deit

Official DeiT repository
Apache License 2.0
3.94k stars 547 forks source link

How long is it supposed to take to train on ImageNet21k for 90 epochs with 8 V100 GPUs #198

Closed Phuoc-Hoan-Le closed 1 year ago

Phuoc-Hoan-Le commented 1 year ago

From DeiT III: Revenge of the ViT (https://arxiv.org/pdf/2204.07118.pdf), how long exactly (exact number of hours) does it take to pretrain for 90 epochs on ImageNet21k with 8 V100 GPUs.

TouvronHugo commented 1 year ago

Hi @CharlesLeeeee, The training time are the following (resolution 224 x 224): Tiny: 2730 s/ epoch (16 GPUs) Small: 1548.6s/ epoch (32 GPUs) Medium: 1577.5 s/ epoch (32 GPUs) Base: 2231.5 s/ epoch (32 GPUs) Large: 5976.10 s / epoch (32 GPUs) Huge: 8319.3s / epoch (64 GPUs)

Best regards, Hugo