search

lukecavabarrett / pna

Implementation of Principal Neighbourhood Aggregation for Graph Neural Networks in PyTorch, DGL and PyTorch Geometric
https://arxiv.org/abs/2004.05718
MIT License
338 stars 55 forks source link

Learnable Scalers #6

Closed ycjing closed 3 years ago

ycjing commented 3 years ago

Hi @lukecavabarrett @gcorso

Thank you for this great work! I have a minor question here. Is it possible to use learnable scalers that could be adapted according to different inputs, instead of using degree-based scalers? Thank you!

Best, Yongcheng

lukecavabarrett commented 3 years ago

Hi @ycjing,

I guess that would be possible, even though I think it might require some important changes to the current implementation. Said that, if you feel like giving it a try, go for it! @gcorso any additional thoughts?

gcorso commented 3 years ago

Hi @ycjing,

Thank you for your interest in our work, I'm not sure I understand your idea, but I'll try to answer, so let me know I misinterpreted your question.

In our work we define the scaler as a function of the degree, that said this function could definitely be parameterized and learned (for example using the parameterization we proposed in our paper and learning alpha with optimization rather than keeping it fixed).

Instead, based on our nomenclature, if you are learning a function of the messages/neighbors you aggregate that is what we called an aggregator. It is indeed possible to parameterize and learn the aggregator and there are a couple of recent works that try this approach.

Finally, I maybe what you were thinking of is instead of doing aggregator * scaler to do the product of two aggregators (where one or both could be learned). This is also possible, but we have not tried it.

Best, Gabriele

ycjing commented 3 years ago

Hi @lukecavabarrett @gcorso

Thank you for your so nice, quick, helpful, and detailed responses! I truly appreciate it! They are really helpful and constructive.

My original intention is trying to use learnable alpha and see if there is any improvement. After reading your kind responses, I am also particularly interested in the mentioned learnable aggregator in the second and third points of @gcorso. Could you please give me the titles of the recent works that try this approach?

I really appreciate your help! Thank you! Wish you all the best!

Best, Yongcheng

gcorso commented 3 years ago

Hi @ycjing,

These are two works I am aware of (although there might be others): https://arxiv.org/pdf/2012.08482.pdf https://arxiv.org/pdf/2006.07739.pdf

Best, Gabriele

ycjing commented 3 years ago

Hi @gcorso

Thank you so much! I truly appreciate your help!

Best, Yongcheng