RobertBiehl / CLIP-tf2

OpenAI CLIP converted to Tensorflow 2/Keras
MIT License
50 stars 6 forks source link

torch>=1.13.1 support, fixed text encoder export #7

Closed RobertBiehl closed 1 year ago

RobertBiehl commented 1 year ago
Leterax commented 1 year ago

Adding tf function decorators to the encode_text and encode_image functions will make them get stored in the saved clip model, might want to add that. For the encode_image function, it doesn't do a lot but the encode_text function has some extra functionality.

Leterax commented 1 year ago

Otherwise looks great, thanks for the quick fix!

RobertBiehl commented 1 year ago

Adding tf function decorators to the encode_text and encode_image functions will make them get stored in the saved clip model, might want to add that. For the encode_image function, it doesn't do a lot but the encode_text function has some extra functionality.

will add that when I'm back

@isaacgerg this should also solve your issues

RobertBiehl commented 1 year ago

@Leterax adding the signatures breaks conversion because the keras weight paths change. So I might have to change some things to make it work. But I'll try it in a followup PR I guess.

RobertBiehl commented 1 year ago

@Leterax adding the signatures breaks conversion because the keras weight paths change. So I might have to change some things to make it work. But I'll try it in a followup PR I guess.

ok signatures for encode_text are in after all.

Text encoder model is also now saved with all the logic of encode_text, not just the transformer backbone