search

lucidrains / lambda-networks

Implementation of LambdaNetworks, a new approach to image recognition that reaches SOTA with less compute
MIT License
1.53k stars 157 forks source link

Implementation of Lambda convolution #18

Closed romulus0914 closed 4 years ago

romulus0914 commented 4 years ago

Thanks for the great work in the implementations!

I would like to ask whether there is a difference in using 'Conv2d' as suggested in Eq. 3 in the paper and your implementation of 'Conv3d'. These two convs treat the (h x w)-dimension as a 1-d sequence and a 2-d image, respectively. I believe they are quite different in concept.

Point out if I misunderstood.

Thx a lot.

lucidrains commented 4 years ago

@romulus0914 thanks! i believe the paper chose a 1d sequence as a simplified example, so people could visualize it better

lucidrains commented 4 years ago

@romulus0914 also, anonymous author reviewed that portion of the code and gave it a green light :) fairly certain it is correct

romulus0914 commented 4 years ago

Thanks for your reply!