How to implement attention mechanism using sparse modules？

facebookresearch / SparseConvNet

Submanifold sparse convolutional networks

https://github.com/facebookresearch/SparseConvNet

Other

2.04k stars 332 forks source link

How to implement attention mechanism using sparse modules？ #219

Open Shaoyi0520 opened 2 years ago

Shaoyi0520 commented 2 years ago

Hello, I am working on a 2-dimensional UNet for sparse image denoising and I would like to integrate the attention mechanism (below) into the UNet network. But I did not succeed, if possible, I would like you to give me some examples of attention for sparse modules, thank you very much! Snipaste_2022-01-25_16-17-29 Snipaste_2022-01-26_16-43-44

bobonpu1 commented 2 years ago

能加个微信吗？

btgraham commented 2 years ago

https://github.com/PointsCoder/VOTR implements sparse attention; maybe that can be useful. But what is the sparse 2D input? And does the sparsity pattern change with denoising?