CarloLucibello
commented
8 months ago Maybe the attention can be implemented through the NNlib's attention function https://github.com/FluxML/NNlib.jl/blob/master/src/attention.jl
Open abieler opened 8 months ago
CarloLucibello
commented
8 months ago Maybe the attention can be implemented through the NNlib's attention function https://github.com/FluxML/NNlib.jl/blob/master/src/attention.jl
Would be fun to have the GPSConv layer in here and I'd be happy to take a crack at it (if deemed useful).
Should be straight forward, except the "global attention" part. Not sure if there is already an implementation somewhere to just add as a dependency (NeuralAttentionlib?) or if implementing it here directly would be the way to go. any thoughts?
https://arxiv.org/abs/2205.12454
https://pytorch-geometric.readthedocs.io/en/latest/generated/torch_geometric.nn.conv.GPSConv.html#:~:text=The%20GPS%20layer%20is%20based,operates%20on%20the%20input%20graph.