How many classes are used for ImageNet to compute `subspace_deepfool()`?

[sayakpaul]

Is it by default 10? What value was used to report the paper results?

[amodas]

Hi @sayakpaul. Yes, the number of classes used by DeepFool is by default 10, which was also used for reporting the results in the paper.

[sayakpaul]

Okay. Thanks. I have a couple more questions:

• Do you have a data-parallel implementation for compute_margin_distribution()? I did use nn.DataParallel but I believe because of the sequential nature of this block nn.DataParallel might not be sufficient.
• I am on GCP with a good enough machine to run the code -- 16 vCPUs with 60 GBs of RAM and 2 V100s. But each batch (size of 128) is taking approximately 120 seconds to finish. We have a total of 391 batches for ImageNet VAL and 217 subspaces. Going with that, each subspace can take a really really long time to compute. Any suggestions to improve it?

[gortizji]

Unfortunately, we do not have a nn.DataParallel implementation as we could not run these experiments in multiple GPUs. However, it should be relatively easy to modify the code to support this feature if you recode DeepFool using a batched implementation. You can get some inspiration from the Foolbox and ART implementations.

Regarding the ImageNet experiments, we do not remember the exact timings, but for the settings in the paper (1,000 samples and a few tens of subspaces, it was less than a day for each network using a single Titan X). In practice, we could not observe any difference in the margin trends when we varied the number of evaluation samples, and hence we never saw the need to run the margin computation in such a large scale.