search

eeyhsong / EEG-Conformer

EEG Transformer 2.0. i. Convolutional Transformer for EEG Decoding. ii. Novel visualization - Class Activation Topography.
GNU General Public License v3.0
398 stars 55 forks source link

Positional Encoding #38

Open schonIs opened 2 days ago

schonIs commented 2 days ago

I am currently conducting research on applying Transformer models to EEG signals. In your paper, I learned that by combining CNN and Transformer, it is possible to learn both global and local features. However, my question is why positional encoding is not added to the data input into the Transformer after the convolution with CNN. Could you please explain the reason for this?

eeyhsong commented 2 days ago

Hello @schonIs,

Happy to see you find this interesting part. I've also tried some position encoding techniques, but there was no significant improvement. I think that's due to the convolution layers performing a role for capturing position information.

Best regards, Song