search

nnzhan / MTGNN

MIT License
798 stars 222 forks source link

Break ties in adjacency matrix construction #26

Closed danielzuegner closed 3 years ago

danielzuegner commented 3 years ago

This PR proposes a fix for the adjacency matrix construction.

I noticed that the learned adjacency matrices look unnatural, with most edges coming out of the first ~50% of the nodes (as per their arbitrary ID). See this example: adj_normal

I have traced this issue back to the combination of torch.topk and tanh. The tanh squashing leads to many entries being (numerically) tied at the maximum possible value of 1.0. In the case of ties, torch.topk always selects the first K tied entries. This leads to the issue in the image above. By adding just a tiny bit amount of random noise, this PR fixes this bias, leading to more "natural" adjacency matrices like this here:

adj_tie_break

nnzhan commented 3 years ago

Hi, sincere thanks for this discovery.