distill.py loss = nan - Githubissues

GeorgeCazenavette / mtt-distillation

Official code for our CVPR '22 paper "Dataset Distillation by Matching Training Trajectories"

https://georgecazenavette.github.io/mtt-distillation/

Other

395 stars 55 forks source link

distill.py loss = nan #6

Closed yangyangtiaoguo closed 2 years ago

yangyangtiaoguo commented 2 years ago

Hello, author. Thank you for your work.！ Running distill During py, loss is always Nan. What parameters do the author suggest to adjust? Or did I ignore what caused the error？ In addition: I use my own dataset. The experimental settings and dataset settings are shown in the figure below.

GeorgeCazenavette commented 2 years ago

Is it NaN loss from the very first iteration?

Is this only happening when you use your own dataset?

Did your experts seem to train okay? If so, around what epoch did they seem to converge?

What model are you using? We used ConvNetD4 for 64x64 images.