The model and pretraining parameters do not match.

[q664171689]

Cause the reason of internet,I have to download the pretrained parameters in local an load it. But it does not match. code:

import openclip import torch model, , preprocess = open_clip.create_model_and_transforms('ViT-H-14-378-quickgelu', pretrained='./DFN5B-CLIP-ViT-H-14-378/pytorch_model.bin')

RuntimeError: Error(s) in loading state_dict for CLIP: Missing key(s) in state_dict: "positional_embedding", "text_projection", "visual.class_embedding", "visual.positional_embedding", "visual.proj", "visual.conv1.weight", "visual.ln_pre.weight", "visual.ln_pre.bias", "visual.transformer.resblocks.0.ln_1.weight", "visual.transformer.resblocks.0.ln_1.bias", "visual.transformer.resblocks.0.attn.in_proj_weight", "visual.transformer.resblocks.0.attn.in_proj_bias", "visual.transformer.resblocks.0.attn.out_proj.weight", "visual.transformer.resblocks.0.attn.out_proj.bias", "visual.transformer.resblocks.0.ln_2.weight", "visual.transformer.resblocks.0.ln_2.bias", "visual.transformer.resblocks.0.mlp.c_fc.weight", "visual.transformer.resblocks.0.mlp.c_fc.bias", "visual.transformer.resblocks.0.mlp.c_proj.weight", "visual.transformer.resblocks.0.mlp.c_proj.bias", .................... . Unexpected key(s) in state_dict: "text_model.embeddings.position_ids", "text_model.embeddings.token_embedding.weight", "text_model.embeddings.position_embedding.weight", "text_model.encoder.layers.0.self_attn.k_proj.weight", "text_model.encoder.layers.0.self_attn.k_proj.bias", "text_model.encoder.layers.0.self_attn.v_proj.weight", "text_model.encoder.layers.0.self_attn.v_proj.bias", "text_model.encoder.layers.0.s .....................

[rwightman]

@q664171689 that's the wrong file, you need the 'open_clip_pytorch_model.bin' ... or use the pretrained functions within open_clip to get the right one.

[q664171689]

Solved,thank you so much.