Hi~ I noticed for the INKD loss in contrastive_training_1.py, the 'imagenet' dataset is used as a fixed auxiliary dataset (Line 602). I am wondering if this means only for the experiment on imagenet, this INKD loss is used, or for all experiments on every dataset, imagenet is always used as auxiliary dataset? The latter does not make sense (maybe my understanding is wrong). Thanks!
Hi~ I noticed for the INKD loss in contrastive_training_1.py, the 'imagenet' dataset is used as a fixed auxiliary dataset (Line 602). I am wondering if this means only for the experiment on imagenet, this INKD loss is used, or for all experiments on every dataset, imagenet is always used as auxiliary dataset? The latter does not make sense (maybe my understanding is wrong). Thanks!