search

mlfoundations / open_clip

An open source implementation of CLIP.
Other
9.93k stars 959 forks source link

Load hybrid-clip in open_clip #863

Closed code-ishwar closed 4 months ago

code-ishwar commented 5 months ago

Is there a way to load a checkpoint from hybrid-clip? I have trained a model on clip-italian. Now I want to finetune using open_clip for stability reasons.

I tried this, but have issues.

rwightman commented 4 months ago

@code-ishwar it can be converted, but keys are different and some layers might be organized a bit differently so needs to be remapped. I'm not aware of any plug and play code going this direction... leaving as discussions in case someone else does