MoCo v3模型跟进 - Githubissues

Oneflow-Inc / libai

LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training

https://libai.readthedocs.io

Apache License 2.0

390 stars 55 forks source link

MoCo v3模型跟进 #144

Closed HiHippie closed 2 years ago

HiHippie commented 2 years ago

记录MoCo v3工作进度

https://arxiv.org/pdf/2104.02057.pdf

[ ] Eager MoCo_v3(Vit)适配及待解决问题：https://github.com/Oneflow-Inc/libai/pull/160
[ ] Loss对齐

rentainhe commented 2 years ago

可以参考 https://github.com/Oneflow-Inc/libai/issues/70 的方式将这个issue细化一下~