search

zju-vipa / Fast-Datafree

[AAAI-2022] Up to 100x Faster Data-free Knowledge Distillation
66 stars 11 forks source link

error when distilling knowledge #2

Open rotem154154 opened 2 years ago

rotem154154 commented 2 years ago

thanks so much for open sourcing your work!

when i train the student from the datafree_kd.py with the parameters in README.md the network is training untill epoch 138 when the if in fast_meta.py line 110 :

if (self.ep == 120+self.ep_start) and self.reset_l0:
   reset_l0(self.generator)

can you explain why reseting the network and why the code crush with the error: AttributeError: 'Generator' object has no attribute 'weight' incide the reset_l0 function: https://github.com/zju-vipa/Fast-Datafree/blob/0955a6cabb60ddb4bf1943d2957bc2ad011f3155/datafree/synthesis/fast_meta.py#L33

Mo-Kanya commented 2 years ago

Hi, Sorry for the late reply. Empirically, we found that resetting part of the generator helps with the diversity of the synthesized samples in the end and (slightly) improves the performance of distillation. The bug (AttributeError:...) has been fixed. Thanks for pointing it out.