search

nateagr / clip-on-yarn

Distributed training of CLIP on Yarn
7 stars 2 forks source link

replication of clip #2

Open rom1504 opened 2 years ago

rom1504 commented 2 years ago

hey, fyi we managed to replicate clip, see https://github.com/mlfoundations/open_clip (will be updated in a few days with the model)

also you may be interested in https://laion.ai/laion-5b-a-new-era-of-open-large-scale-multi-modal-datasets/ and https://rom1504.medium.com/semantic-search-at-billions-scale-95f21695689a where you might find a similarity with the image text pipeline ;)

rom1504 commented 2 years ago

I hope you manage to get a criteo-clip as well :)

nateagr commented 2 years ago

Hello @rom1504 ! Very interesting articles :) Good job ! Yes on our side, I finally had time to focus for few hours on the problem we had with the distributed training of CLIP and fixed the issue. We are now able to fine-tune the model without issue.