Working with Edge Features

[flogrammer]

Hello,

as far as I understand it the library allows to distuingish between differen edge types but cannot incorporate continuous edge features. But especially when working with molecules, adding bond information to the network adds a lot of value.

So far in tf2-gnn, a message passing layer recieves the information about node_features and adjacency_lists. If I want to extend the library I would have to additionally pass the edge_features to the respective layers. Additionally, the separate treatment of edge types would be obsolete, as they could simply be encoded in the feature vectors.

Now one workaround would be to include the edge features of all neighboring edges to the node features of a specific node. I tried this and the results were not much different from simply using plain node features. Concretely, I want to implement this paper, through which I hope to improve my results.

Long story short, do you plan to extend the library for edge features in the future, as I assume that this will lead to a lot of architectural changes.

Thanks

[kmaziarz]

Now one workaround would be to include the edge features of all neighboring edges to the node features of a specific node. I tried this and the results were not much different from simply using plain node features.

Another way to insert edge features would be to create a virtual node in the middle of every edge. One consequence though would be doubling the graph diameter.

Additionally, the separate treatment of edge types would be obsolete, as they could simply be encoded in the feature vectors.

Theoretically yes. I'm not sure what would be the effect in practice though - I guess having an explicit separation of weights corresponding to different edge types may work better than using a deeper MLP that would additionally take in edge types encoded as one-hot.

[mmjb]

Hey,

Sorry for the late reply (also on the other ticket) - for some reason I wasn't watching this repo.

If I want to extend the library I would have to additionally pass the edge_features to the respective layers.

I actually had experimental implementation of that lying around that I've now just pushed (cf. branch dev/mabrocks/edge_feats). I have more code that actually uses this (as an extension to work with the ogb datasets) and will try to clean that up and push ASAP, so that you can see how to use this.

Additionally, the separate treatment of edge types would be obsolete, as they could simply be encoded in the feature vectors.

This is not the full story here, I fear. In theory, using a deeper network for the message computation + encoding edge type as a feature would be as powerful as having separate weights. However, in practice, such approaches have a number of issues (e.g., the deeper edge networks cause substantially higher memory use, as memory is dominated by the message computation; you need to invest more compute; it's surprisingly hard to optimise). In my experience, having separate message functions for separate edge types is quite useful, at least if the number of edge types is low.

Marc

[flogrammer]

Hi Marc and kmaziarz,

thanks for the replies!

Will definitely have a look at your experimental implementation - thanks a lot!

Best regards and a good time

[flogrammer]

By the way, I implemented this paper for GAT: https://openaccess.thecvf.com/content_CVPR_2019/papers/Gong_Exploiting_Edge_Features_for_Graph_Neural_Networks_CVPR_2019_paper.pdf

However I experienced, that carrying the edge features as a Tensor of shape No.Nodes x No.Nodes x EdgeFeatureVecSize is quite tedious and significantly reduces the training time.