yanghu819 / Graph-MLP

73 stars 21 forks source link

NAN loss problem[SOLVED]: Performance Plummets When Setting adj_label Diagonal to Zero #1

Closed ChildishBob closed 3 years ago

ChildishBob commented 3 years ago

After setting the diagonal of adj_label to zero: image

The test accuracy for cora goes down to 0.13 with the same configuration: image

which is quite confusing since this modification is in accordance with: image

yanghu819 commented 3 years ago

Thank you so much for your issue on the NAN loss in training:

We fix the bug, for the code will bring NAN loss in training because "log(0)" in image

We add 1e-8 to prevent the NAN loss. Now the code is updated, please try it.

Also, you don't have to add the code for we remove the distance of its self in get_feature_dis in models.py image

ChildishBob commented 3 years ago

Thanks for your work! By the way, you said

Also, you don't have to add the code for we remove the distance of its self in get_feature_dis in models.py

However, in image

even when x_dis is 0, we have e^0 = 1. That conflicts with image

yanghu819 commented 3 years ago

I think you did a very good job in finding this minute difference. That indeed will result a bit difference in formula,but the loss function optimized won' t be influenced by this operation: an added constant value will not influence the gradient much, also, you can try to change it into a better formula which can be far better than this demo. In fact, even removing the mask to avoid computing loss function of loss_ii won't do harm to the final results.