Outputs in autoencoder.py

[hkim716]

❓ Questions & Help

I'm trying to use autoencoder.py in the example folder for my practice. Let say I would use model = GAE(LinearEncoder(num_features, out_channels)) as my GNN model. When I'm thinking of a deep learning model with image data such as 28x28 pixel data, autoencoder loss is defined as loss=MSE(outputs from decoder, inputs).

However, here in the graph neural network autoencoder, it was defined as loss = model.recon_loss(z, train_pos_edge_index) Why it does not compare input x with the outputs from the decoder (I guess decoder was defined as InnerProductDecoder() in GAE) ? and what does InnerProductDecoder() actually work as a decoder? I think it is just matrix multiplication of edge_index... how is this simply defined?

When I try to see the outputs from decoder, I wrote reconstructed = model.decode(z, train_pos_edge_index) and checked the shape of it, and it was the same shape of train_pos_edge_index.shape[1]. Is it correct way to see reconstructed outputs from the decoder to compare input x? I'm confusing.. please help me.

Thanks!

[rusty1s]

The GAE model does not reconstruct input node features, it reconstructs adjacency information. It is a link prediction example. Therefore, model.decode(z, train_pos_edge_index) will get you the probability for each edge in train_pos_edge_index.

[hkim716]

Thanks Matt for your reply. I was wondering what would you suggest in case our adjacency matrix is fixed, and we are trying to predict P(X|Z) rather than P(A|Z), would that require a lot of modifications? or have you seen any example for that case?

[rusty1s]

This problem is actually easier to implement, as it is similar to a classic autoencoder. You simply pass your node features into a bottleneck GCN, from which you try to reconstruct input node features, e.g.:

conv1 = GCNConv(in_channels, 64)
conv2 = GCNConv(64, 16)  # <-- bottleneck
conv3 = GCNConv(16, 64)
conv4 = GCNConv(64, in_channels)

y = model(x, edge_index)
loss = F.mse_loss(x, y)

[liyuntong9]

@rusty1s Hi Matt, I have a similar question for you.

When using GCN encoder, we need to aggregate neighbor information, that is, the features of neighbor nodes. The encoded input should be the low dimensional representation of node features. However, why is the adjacency matrix reconstructed from the latent representation of nodes?

[rusty1s]

The autoencoder example does not try to encode node features but graph structure information instead, i.e. it learns similar embeddings for nodes that might be connected.

[liyuntong9]

@rusty1s Thanks for your reply. Your code about GAE class is exactly the same as the formula mentioned in the paper "Variational Graph Auto-Encoders", and You use this forward_all function to get the reconstructed adjacency matrix. However, what is the meaning of the return value of this function, and how can I use it to reconstruct the real graph？

[liyuntong9]

At first, I think this matrix represents the probability of whether two nodes have edges. And then I use the part of the matrix value greater than 0.85 to reconstruct the whole graph. I was surprised to find that this reconstructed graph has 270241 edges, while the Cora dataset has only 10556 edges. So I don't understand the meaning of the return value of this function forward_all.

[rusty1s]

That is correct. The GAE model is far from perfect, in particular because it uses a non-trainable decoder. In addition, its task is to find potentially positive edges outside of the training set, not to perfectly reconstruct all training edges.

[liyuntong9]

That is correct. The GAE model is far from perfect, in particular because it uses a non-trainable decoder. In addition, its task is to find potentially positive edges outside of the training set, not to perfectly reconstruct all training edges.

Ah, I understand. Thank you again for your explanation.

[ved164]

@rusty1s Can you suggest some literature or implementation where we are trying to learn the node features instead of the adjacency matrix? Thank you.

[rusty1s]

The general procedure for encoding or learning masked out node features is the same as for any autoencoder, so there does not really exist a designated approach in the graph learning community. This involves, embedding nodes into a low-dimensional space and then decoding them again. You can use masks to learn the node features of unseen nodes. Let me know if that makes sense!