Closed DomInvivo closed 4 years ago
Out of curiosity, can you elaborate on the type of dataset you have found this to be more effective on? # molecules, information about task, etc.
I am working on a molecular dataset for mRNA signature similarity matching, with the dataset provided by the following GitHub: deepSIBA.
After careful parameter optimization, I found that the simpler architecture performs similarly to the MPNN-based architecture on this dataset but with considerably fewer features since the towers are not required. I also found that edge features decreased model performance, which I find very weird.
Added, thank you very much Dominique!
I found for some tasks that I'm working on personally that the MPNN-style architecture does not perform well, no matter the aggregators or scalers that are used. Even the simple
sum
andmean
aggregators perform less well than their GIN and GCN cousins.For this reason, I propose to add the following simpler architecture as a variant of the PNA layer, which doesn't use the MPNN attention mechanism, but instead aggregates the neighbours in a similar way than CNN, GCN and GIN layers. An obvious drawback is the lack of edge features, but on my personal project on a molecular dataset, edge features seem to cause more overfit.
I propose to add it in the file
pna/models/dgl/pna_layer.py
. I did not implement it in pytorch-geometric or standard pytorch.