search

iancovert / Neural-GC

Granger causality discovery for neural networks.
MIT License
198 stars 51 forks source link

cMLP: GC_est's weights show 'nan' #3

Closed KeSyren closed 3 years ago

KeSyren commented 4 years ago

Hi authors,

I ran the linear simulation data with p=30 using cMLP. The GC_est shows some 'nan's. Could you explain why and how to fix it?

iancovert commented 3 years ago

Hi, sorry for the slow response. If the GC_est weights show nan values, it must mean that the network itself has nan values in the parameters. It's a bit surprising to have this happen during training, because there aren't any logarithms or divisions that could be hitting zeros or negative numbers... The proximal steps involve division, but the functions that perform those steps are careful to avoid dividing by zero.

In my experience, the only way I've seen nan values during training is when I set the learning rate too high, so that's my best guess. If you're still looking into this, you might try lowering the learning rate or switching to training with ISTA rather than GISTA (see the updated notebooks). In an updated version of our paper we're going to be recommending ISTA rather than GISTA because it's faster and the results are just as good (although ISTA requires a bit more learning rate tuning because it has no line search).