Closed qfmy closed 1 month ago
@qfmy it's not normal and it does not occur for me. Are you sure there isn't an issue on your setup loading the pretrained weights? corrupt file download?
If you happened to have enabled stochastic depth or dropouts, you need model.eval() after creation, just updated the README
Everytime I run the demo below,the result is random. Like:
first time: Label probs: tensor([[0.1304, 0.1254, 0.7442]]) second time: Label probs: tensor([[0.0421, 0.4105, 0.5474]]) third time: Label probs: tensor([[0.0628, 0.8441, 0.0931]])
Is this normal?
import torch from PIL import Image import open_clip
model, _, preprocess = open_clip.create_model_and_transforms('ViT-B-32', pretrained='laion2b_s34b_b79k') tokenizer = open_clip.get_tokenizer('ViT-B-32')
image = preprocess(Image.open("docs/CLIP.png")).unsqueeze(0) text = tokenizer(["a diagram", "a dog", "a cat"])
with torch.no_grad(), torch.cuda.amp.autocast(): image_features = model.encode_image(image) text_features = model.encode_text(text) image_features /= image_features.norm(dim=-1, keepdim=True) text_features /= text_features.norm(dim=-1, keepdim=True)
print("Label probs:", text_probs) # prints: [[1., 0., 0.]]