search

cfifty / CAML

MIT License
53 stars 8 forks source link

Some minor errors in src/evaluation/test.py which makes running with ViT-H laion2b fall #12

Closed Farenweh closed 2 months ago

Farenweh commented 3 months ago

if args.model == 'CAML': if 'clip' in args.fe_type: model_path = '../caml_pretrained_models/CAML_CLIP/model.pth' elif 'laion' in args.fe_type: model_path = '../caml_pretrained_models/CAML_Laion2b/model.pth'

The code at the beginning of src/evaluation/test.py mentioned above will cause ViT-H laion2b to be unavailable because its timm model name is "timm:vit_huge_patch14_clip_224.laion2b:1280", which contains "clip". This will cause it to enter the wrong conditional branch, MEANWHILE, w/o setting another encoder_size from the default one (it's base) , a wrong encoder_size which leads to a dimenson mismatch later in Encoder.

The solution is to change the above code to: if "openai" in args.fe_type: model_path = "../caml_pretrained_models/CAML_CLIP/model.pth" encoder_size='large' elif "laion" in args.fe_type: model_path = "../caml_pretrained_models/CAML_Laion2b/model.pth" encoder_size='laion' # 对应TransformerEncoder.py line 36

cfifty commented 2 months ago

Hey! Thanks for the heads up -- it's now fixed.