search

lucidrains / VN-transformer

A Transformer made of Rotation-equivariant Attention using Vector Neurons
MIT License
83 stars 4 forks source link

Expected release date #1

Closed ZaiyunLin closed 1 year ago

ZaiyunLin commented 2 years ago

Hi lucidrains, is there an expected release date for this project. I am currently implementing the VN-transformer as well, but facing some technical difficulties.

dennisushi commented 1 year ago

Hi lucidrains, is there an expected release date for this project. I am currently implementing the VN-transformer as well, but facing some technical difficulties.

Did you manage to implement it?

lucidrains commented 1 year ago

@dennisushi @Anonnoname hey, sorry for getting back to this project so late

going to finish this up by latest next week's end

lucidrains commented 1 year ago

seems like there's a lot of interest in it!

lucidrains commented 1 year ago

also, just so you are all aware, i think there's been kind of a small breakthrough in the equivariance space https://arxiv.org/abs/2302.03655

lucidrains commented 1 year ago

ok, almost there, save for the early or late fusion of non-spatial features

lucidrains commented 1 year ago

ok, it is done