search

wzlxjtu / PositionalEncoding2D

A PyTorch implementation of the 1d and 2d Sinusoidal positional encoding/embedding.
251 stars 24 forks source link

The source of the 2D positional encoding #1

Open RuijieJ opened 5 years ago

RuijieJ commented 5 years ago

Hi, I think this work for 2D positional encoding is beautiful and helpful for when dealing with image data. I wonder if the 2D positional encoding is your original work or from a published paper? Thanks a lot!

wzlxjtu commented 5 years ago

Hi Ruijie, I haven't seen this work being published anywhere yet. This work is part of our larger project, which we expect to publish on ArXiv in about 2 weeks. For now, you can refer to the 1D version in "Attention Is All You Need". You are welcome to use the code :)

RuijieJ commented 5 years ago

Thank you! The work is brilliant and I may use it in my future project. I will be appreciate if you inform me when your paper are published so that I can cite it.

wzlxjtu commented 5 years ago

Hi Ruijie, the paper is finally out. It's available as a preprint at https://arxiv.org/abs/1908.11415