This paper has been accepted by CVPR 2024. We are profoundly grateful for the significant insights provided by CLIPN.
Open-World Few-Shot Learning (OFSL) is a critical field of research, concentrating on the precise identification of target samples in environments with scarce data and unreliable labels, thus possessing substantial practical significance. Recently, the evolution of foundation models like CLIP has revealed their strong capacity for representation, even in settings with restricted resources and data. This development has led to a significant shift in focus, transitioning from the traditional method of “building models from scratch” to a strategy centered on “efficiently utilizing the capabilities of foundation models to extract relevant prior knowledge tailored for OFSL and apply it judiciously”. Amidst this backdrop, we unveil the Direct-and-Inverse CLIP (DeIL), an innovative method leveraging our proposed “Direct-and-Inverse” concept to activate CLIP-based methods for addressing OFSL. This concept transforms conventional single-step classification into a nuanced two-stage process: initially filtering out less probable categories, followed by accurately determining the specific category of samples. DeIL comprises two key components: a pre-trainer (frozen) for data denoising, and an adapter (tunable) for achieving precise final classification.
pip install -r requirements.txt
main_path
in the main.py file on line 26 to match the dataset you intend to validate.main_path
to main_path = "./configs/imagenet/config.yaml"
root_path
on the 2nd line of the config.yaml
file corresponding to your dataset../configs/imagenet/config.yaml
file, update the root_path
to root_path: "./DATA/"
CUDA_VISIBLE_DEVICES=0 python main.py
This repo benefits from CLIPN, CaFo, CLIP, and DALL-E. Thanks for their wonderful work.
@inproceedings{shao2024deil,
title={DeIL: Direct-and-Inverse CLIP for Open-World Few-Shot Learning},
author={Shao, Shuai and Bai, Yu and Wang, Yan and Liu, Baodi and Zhou, Yicong},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
pages={28505--28514},
year={2024}
}
@inproceedings{shao2024collaborative, title={Collaborative Consortium of Foundation Models for Open-World Few-Shot Learning}, author={Shao, Shuai and Bai, Yu and Wang, Yan and Liu, Baodi and Liu, Bin}, booktitle={Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)}, volume={38}, number={5}, pages={4740--4747}, year={2024} }