search

NVlabs / DeepInversion

Official PyTorch implementation of Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion (CVPR 2020)
Other
479 stars 77 forks source link

How to search for hypers on a new dataset #14

Closed ericxian1997 closed 1 year ago

ericxian1997 commented 3 years ago

Hi, thanks for your inspiring work and the generated ImageNet images are awesome. However, the generated images doesn't look good when I replace the ImageNet model with a model trained on another dataset. It seems the quality of the images is sensitive to the hypers ('tv_l1','tv_l2','r_feature' and 'l2'). Could you provide any insight of searching good hypers on a new dataset?

hongxuyin commented 3 years ago

Checking the corresponding losses can be helpful. R_feature is the most important and it can be very beneficial to start on it, find a valid alpha for it that gives valid feature distribution loss as printed out by code (e.g. in loss range 1-10). Then, gradually add in tv and l2 for improvements.