search

rashindrie / DIPA

Discriminative Sample-Guided and Parameter-Efficient Feature Space Adaptation for Cross-domain Few-Shot Learning - CVPR 2024
MIT License
10 stars 2 forks source link

Some minor questions #1

Closed yiyuyuyi closed 2 months ago

yiyuyuyi commented 4 months ago

Hello! Thank you for your inspiring open-source work! I have a few questions regarding your work and code and would greatly appreciate your assistance in answering them:

  1. How much GPU memory is approximately required to complete a full Meta-Dataset test?
  2. I noticed in the ablation study attached to your paper that the Nadam optimizer yields a 3% improvement over the AdamW optimizer on the "Average all" metric. To replicate this experiment, should the learning rate and other settings for AdamW be consistent with those used for the Nadam optimizer?
  3. When pretraining MDL using iBoT, how is the all_paths_{mode}.json file generated? Is it created by consolidating all data paths from the eight datasets into all_paths_{mode}.json? If so, how do you avoid sampling a large number of samples from the QuickDraw dataset (which contains significantly more samples than the other datasets) during data sampling?

I am looking forward to your response! Thank you!

rashindrie commented 2 months ago

Hello @yiyuyuyi ,

Thank you for your interest in our work.

  1. We used only one A100 GPUs for a round of testing.
  2. Yes, keep everything else the same and change only the optimizers.
  3. Good observation. I had to create the 'all_paths' files using a custom code prior to pre-training. In that code, I limit the number of samples from the QuickDraw dataset as this dataset contains too many files. I noticed that if I simply use all the samples from QuickDraw for pre-triaining, iBoT fails as there is a very high imbalance of images.

Hope these answers are of help for you.

Thanks, Rashindrie