search

facebookresearch / moco-v3

PyTorch implementation of MoCo v3 https//arxiv.org/abs/2104.02057
Other
1.22k stars 161 forks source link

How many TPUs ? #22

Closed jrabary closed 2 years ago

jrabary commented 2 years ago

Hi, In the moco-v3 paper there is a section about the computation time. It says that for the ViT-B, 100 epochs of imagenet take 2.1h hours. It is not clear if it 512 TPU devices or 512 TPU cores. To be precise, there are two types of TPUs available on google cloud: v2-[32,512] and v3-[32-2048]. Which one of them was used in the experiment and how many for each instance ?

endernewton commented 2 years ago

v3 is generally used in our experiments. When it is 512, it means 512 TPU cores, so v3-512.