Implementation of HNN++ Poincaré multi-head attention - Githubissues

maxvanspengler / hyperbolic_learning_library

An extension of the PyTorch library containing various tools for performing deep learning in hyperbolic space.

MIT License

134 stars 9 forks source link

Implementation of HNN++ Poincaré multi-head attention #56

Open maxvanspengler opened 9 months ago

maxvanspengler commented 9 months ago

The HNN++ paper introduced a multi-head attention mechanism within the Poincaré ball. Most of the important operations are already available within the library except for the similarity functions. So to implement this we probably need:

The layer itself (based on HNN++ and the original PyTorch MultiheadAttention layer);
Any relevant similarity functions on the Euclidean and PoincareBall manifolds;
Tests to check the above;
Some basic documentation.