GAT outputs same embedding for all nodes if defined on complete graph

[velix]

I am trying to train a 3 layer GAT network with multiple heads on a complete graph of 100 nodes. Each node represents a point in space, so the node's features are its coordinates in [0,1]. The output of the network is the same embedding for each of the nodes, a (128, 100) matrix with each column identical to the rest.

A toy example

using GeometricFlux, LightGraphs, Flux

nodes_c = 10
node_features = rand(2, nodes_c)

g = LightGraphs.complete_graph(nodes_c)

model = Chain(GATConv(g, 2=>4), GATConv(g, 4=>8), GATConv(g, 8=>16))

model(node_features)

which outputs the same 16d embedding for each node.

Am I misunderstanding something about GATs here, or does the GATConv implementation not work with complete graphs?

[yuehhua]

A complete graph may not work with graph attention network. A node i is used to query its neighbors in a graph attention network. However, a complete graph has every node be the neighbor of all other nodes. The query happens to each node over the rest nodes. This lead to the same embedding for each node.

[yuehhua]

If you have any questions, it's welcome to reopen the issue.