In GATConv an GATv2Conv the softmax calculation for determining the attention coefficients can too easily lead to overflows, e.g., already with input values of ~100. It should be realized in a numerically more stable way, e.g ,. as in AGNNConv by using softmax_edge_neighbors. Most importantly, a robust softmax implementation subtracts the maximum before exponentiating.
In
GATConv
anGATv2Conv
the softmax calculation for determining the attention coefficients can too easily lead to overflows, e.g., already with input values of ~100. It should be realized in a numerically more stable way, e.g ,. as inAGNNConv
by usingsoftmax_edge_neighbors
. Most importantly, a robust softmax implementation subtracts the maximum before exponentiating.