search

navervision / KELIP

Official PyTorch implementation of "Large-scale Bilingual Language-Image Contrastive Learning" (ICLRW 2022)
Apache License 2.0
95 stars 11 forks source link

[BUG]build_model loads wrong state_dict #2

Open Dango233 opened 2 years ago

Dango233 commented 2 years ago

Short in short, in Windows, when initiating model using kelip.build_model('ViT-B/32'), sometimes it loads a wrong state_dict from the huggingface cache folder. Maybe I've downloaded too much stuff from Huggingface, especially many different clips?

Now I have to add a line to load the right state_dict: model.load_state_dict(torch.load(r"C:\Users\*****\.cache\huggingface\transformers\8f525db77e15271b26656a43682a71b06f93efef9b38f2e880117210d3f3be74.f30d465dc319e53701f2bea7f3e60214c3846cd9cc0b6b27445f121d496322f6"))

Maybe rename the model will help?

geonm commented 2 years ago

No. It won't solve the problem you reported.

'ViT-B/32' is used to indicate the name of revision, not the whole model name.

Please check this line: https://github.com/navervision/KELIP/blob/master/kelip/model_loader.py#L108-L111

We didn't try to test KELIP with any edition of Windows.

kobiso commented 2 years ago

@Dango233 I had the similar problem in Ubuntu, and I fixed it by removing the huggingface cache folder and re-downloading the model checkpoints. Can you try to remove the cache folder? I haven't tried in Windows, tho.