facebookresearch / ijepa

Official codebase for I-JEPA, the Image-based Joint-Embedding Predictive Architecture. First outlined in the CVPR paper, "Self-supervised learning from images with a joint-embedding predictive architecture."
Other
2.83k stars 358 forks source link

Support for GPUs that can't use bfloat16 and non-distributed training #28

Open JulesCollenne opened 1 year ago

JulesCollenne commented 1 year ago

Parts of the code still used bfloat16 even when "use_bfloat" was False. Added a condition to check if distributed learning is possible or not depending on the world size.