about the kernel size - Githubissues

SHI-Labs / Neighborhood-Attention-Transformer

Neighborhood Attention Transformer, arxiv 2022 / CVPR 2023. Dilated Neighborhood Attention Transformer, arxiv 2022

MIT License

1.04k stars 85 forks source link

about the kernel size #35

Closed XiaoyuShi97 closed 2 years ago

XiaoyuShi97 commented 2 years ago

Hi, I find that there is no explicit parameter specifying kernel size in natten.py. How does cuda code get it? By the shape of rpb? Thanks!

alihassanijr commented 2 years ago

Hello, thanks for your interest. Yes, RPB is always in the shape of 2 * kernel_size - 1 across the two axes, for every head, therefore it's easy to obtain kernel_size from that.

XiaoyuShi97 commented 2 years ago

Thanks for your prompt reply!

alihassanijr commented 2 years ago

Closing this due to inactivity. If you still have questions feel free to open it back up.