Open BlinkDL opened 2 years ago
@BlinkDL Hey Peng Bo! So I quickly checked the script and indeed it NaNs
, but not if the visual_ssl
is turned off
I suspect it has something to do with augmenting the randomly created images in the visual SSL, but not completely sure
Hi lucidrains,
Try this and it will NaN within 100 steps (latest Github code). The loss looks fine before NaN.