GCNConv: Weighted Self-Loops, Gaussian Initialization

[GageDeZoort]

🚀 The feature, motivation and pitch

I'd like to propose the implementation of a new GCNConv using recommendations from this paper:

Principles for Initialization and Architecture Selection in Graph Neural Networks with ReLU Activations The relevant modifications include:

• The linear layer self.lin would have a Gaussian initialization scheme with no biases and weights sampled from a zero-mean Gaussian with variance C_w/self.in_channels, where C_w is set just above 2 by default.
• In the gcn_norm function, allow for self-loops to contribute more in proportion to weights calculated via edge_weight = deg_inv_sqrt[row] * edge_weight * deg_inv_sqrt[col] (even if self loops are already included in deg_inv_sqrt). Specifically, if the edge_weight values are stored in a weighted adjacency matrix P, message passing would be (1-t) identity + t P for a hyperparameter 0<t<1.

In summary, this would simply generalize the GCNConv to include hyperparameters C_w and t.

Alternatives

No response

Additional context

No response

[rusty1s]

This sounds cool :) We can either add this to the original GCNConv layer or in a separate module that inherits from it. What do you think?

[GageDeZoort]

Thanks for the response, Matthias :) I think it'd be nice to have t and C_W as options in the original GCNConv layer. I'd be fine with having a separate module, though - this is only a loose preference. Either way, I'm happy to help with any implementation or surrounding documentation!

[rusty1s]

Super. Would it be possible for you to provide the initial implementation directly in GCNConv, and I can take it from there?

[GageDeZoort]

Sure, I'm happy to - will open a PR in the next few days!