search

NVlabs / DeepInversion

Official PyTorch implementation of Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion (CVPR 2020)
Other
479 stars 77 forks source link

Having trouble installing apex and getting the result for the basic snippet #7

Closed haripriyaharikumar closed 3 years ago

haripriyaharikumar commented 3 years ago

Hi, After running this basic snippet (which I think does not require apex installation) python imagenet_inversion.py --bs=84 --do_flip --exp_name="rn50_inversion" --r_feature=0.01 --arch_name="resnet50" --verifier --adi_scale=0.0 --setting_id=0 --lr 0.25, I am getting only noisy images in the /generations folder and the verifier accuracy is only 0.0%. No final result in the final_images folder. I am having trouble installing apex as well.

hongxuyin commented 3 years ago

The snippet can generate image batch without AMP but 2x faster once installed. The verifier accuracy will start to increase once the losses begin to drop after several hundreds of itrs, and images will be saved accordingly. AMP installation can be found here https://github.com/NVIDIA/apex.

haripriyaharikumar commented 3 years ago

okay, Thanks. I have another question about the student network you used in the CIFAR-10 experiments. Are they trained on any data or not trained on any data? Bit confusing when it comes to the adaptive deep inversion where you find the most dissimilar images between student and teacher networks.

hongxuyin commented 3 years ago

For CIFAR-10, all students start as reinitialized models pretrained on no data. ADI is iterative given intermediate students during optimization at specs in the paper. An initial set of DI images are generated to kick start the KD process, where ADI samples are newly generated iteratively during KD and then mixed with original sets.