Closed mehdidc closed 1 year ago
Supports a generic way to use custom prompts / templates / classnames. Default stay the same as currently, with the addition of the possibility to provide a custom prompts / templates and classname files.
e.g. clip_benchmark eval --dataset "wds/imagenet1k" --model ViT-B-32 --pretrained laion400m_e32 --custom_template_file <path> --custom_classname_file <path>
clip_benchmark eval --dataset "wds/imagenet1k" --model ViT-B-32 --pretrained laion400m_e32 --custom_template_file <path> --custom_classname_file <path>
the template file can be either in the usual format https://github.com/LAION-AI/CLIP_benchmark/blob/main/clip_benchmark/datasets/en_zeroshot_classification_templates.json or in the CuPL format https://github.com/LAION-AI/CLIP_benchmark/blob/main/clip_benchmark/datasets/cupl_prompts.json to have class-specific prompts. In the case of CuPL format, the classnames file will not be used.
Supports a generic way to use custom prompts / templates / classnames. Default stay the same as currently, with the addition of the possibility to provide a custom prompts / templates and classname files.
e.g.
clip_benchmark eval --dataset "wds/imagenet1k" --model ViT-B-32 --pretrained laion400m_e32 --custom_template_file <path> --custom_classname_file <path>the template file can be either in the usual format https://github.com/LAION-AI/CLIP_benchmark/blob/main/clip_benchmark/datasets/en_zeroshot_classification_templates.json or in the CuPL format https://github.com/LAION-AI/CLIP_benchmark/blob/main/clip_benchmark/datasets/cupl_prompts.json to have class-specific prompts. In the case of CuPL format, the classnames file will not be used.