Loading custom clip

[anilsathyan7]

from fashion_clip.fashion_clip import FashionCLIP
fclip = FashionCLIP('/content/clip.pth')

NameError                                 Traceback (most recent call last)
[<ipython-input-3-ca60ff7e5359>](https://localhost:8080/#) in <cell line: 3>()
      1 from fashion_clip.fashion_clip import FashionCLIP
      2 
----> 3 fclip = FashionCLIP('/content/finetuned_clip.pt')

1 frames
[/usr/local/lib/python3.10/dist-packages/fashion_clip/fashion_clip.py](https://localhost:8080/#) in _load_model(self, name, device, auth_token)
    170                 raise RuntimeError(f"Model {name} not found or not valid; available models = {list(_MODELS.keys())}")
    171 
--> 172             model, preprocessing = clip.load(model_path, device=device, download_root=_CACHE_DIR)
    173             hash = file_sha256(model_path)
    174 

NameError: name 'clip' is not defined

Unable to load custom /openai clip. (https://github.com/openai/CLIP) Is it possible to load open_clip models also (https://github.com/mlfoundations/open_clip)?

[vinid]

Hi!

FashionCLIP mainly uses the HuggingFace interface for models, so it would be better to use.

FashionCLIP('openai/clip-vit-base-patch32')

or a model saved in that form.

If you want to use CLIP-format you need to install the clip package form openai

[anilsathyan7]

# else it doesn't use HF, assume using OpenAI CLiP
        else:
            if os.path.isfile(name):
                model_path = name
            elif validators.url(name):
                # generic url or S3 path
                model_path = _download(_MODELS[name], _CACHE_DIR)
            else:
                raise RuntimeError(f"Model {name} not found or not valid; available models = {list(_MODELS.keys())}")

            model, preprocessing = clip.load(model_path, device=device, download_root=_CACHE_DIR)

Ok, in the codebase it there was a file path reference in '_load_model'.

[vinid]

sorry, do you mind giving me additional details regarding your request?

[anilsathyan7]

Can we load a model from a path to a local checkpoint(pytorch)?

[vinid]

Yes, but it needs to be in the correct format, best if it is an HF checkpoint