openai / CLIP

CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image
MIT License
26.23k stars 3.35k forks source link

Attempt to load from an already-used file pointer in clip/clip.py #396

Open mrd opened 1 year ago

mrd commented 1 year ago

https://github.com/openai/CLIP/blob/a1d071733d7111c9c014f024669f959182114e33/clip/clip.py#L136

Went through this code path when attempting to load my own saved checkpoint, and kept getting an EOF error until I inserted a rewind, opened_file.seek(0) before the torch.load (linked above).

This is because the try...except block has a (different) load in the try part, so if it falls back to the exception handler the file-handle is not necessarily at the beginning of the file, and therefore it is not valid to try to load from the file-handle again as-is. It is possible that it would be better to separate the exception handling for file errors from the exception handling for 'jit loading problems', or maybe even better not to try to load with the jit module when the jit option is False, but I'll leave that to you to decide.

Note that I have since switched to loading the ViT-B32 model and then replacing the state_dict, so I don't trigger this bug in my own code anymore.