search

facebookresearch / moco-v3

PyTorch implementation of MoCo v3 https//arxiv.org/abs/2104.02057
Other
1.2k stars 158 forks source link

Tensorflow version #8

Closed wildphoton closed 3 years ago

wildphoton commented 3 years ago

Thank you for open source the Pytorch implementation. I wonder if the original tensorflow implementation have been released for the purpose of training on TPUs.

endernewton commented 3 years ago

We have not released and there is no plan to release the code. This implementation is supposed to be a faithful re-implementation of the TPU code, with details such as initialization (though still subject to framework differences).

endernewton commented 3 years ago

The mmf team (https://github.com/facebookresearch/mmf), specifically @ronghanghu have a good implementation of MoCo v3 using PyTorch XLA, which could be a good starting point on TPUs. Maybe good to check about their plan for releasing it.

wildphoton commented 3 years ago

Thanks!

ronghanghu commented 3 years ago

@wildphoton I can share with you a copy of our TPU implementation w/ PyTorch XLA if you are interested.

wildphoton commented 3 years ago

@wildphoton I can share with you a copy of our TPU implementation w/ PyTorch XLA if you are interested.

Sure. It would be great to have access to it!

MukundVarmaT commented 2 years ago

hi @ronghanghu @wildphoton, Can you please share the TPU implementation w/ torch xla? If it's already hosted on a different repo, could you please point me to the same?

Thanks in advance.

ronghanghu commented 2 years ago

@MukundVarmaT it's in https://github.com/ronghanghu/simclr_vit_profiling/tree/mocov3_vit_release