Open GageDeZoort opened 1 year ago
This sounds cool :) We can either add this to the original GCNConv
layer or in a separate module that inherits from it. What do you think?
Thanks for the response, Matthias :) I think it'd be nice to have t
and C_W
as options in the original GCNConv
layer. I'd be fine with having a separate module, though - this is only a loose preference. Either way, I'm happy to help with any implementation or surrounding documentation!
Super. Would it be possible for you to provide the initial implementation directly in GCNConv
, and I can take it from there?
Sure, I'm happy to - will open a PR in the next few days!
🚀 The feature, motivation and pitch
I'd like to propose the implementation of a new
GCNConv
using recommendations from this paper:Principles for Initialization and Architecture Selection in Graph Neural Networks with ReLU Activations The relevant modifications include:
self.lin
would have a Gaussian initialization scheme with no biases and weights sampled from a zero-mean Gaussian with varianceC_w/self.in_channels
, whereC_w
is set just above 2 by default.gcn_norm
function, allow for self-loops to contribute more in proportion to weights calculated viaedge_weight = deg_inv_sqrt[row] * edge_weight * deg_inv_sqrt[col]
(even if self loops are already included indeg_inv_sqrt
). Specifically, if theedge_weight
values are stored in a weighted adjacency matrix P, message passing would be (1-t
) identity +t
P for a hyperparameter 0<t
<1.In summary, this would simply generalize the
GCNConv
to include hyperparametersC_w
andt
.Alternatives
No response
Additional context
No response