search

Vladimir2506 / Pointformer

Repository of 3D Object Detection with Pointformer (CVPR2021)
155 stars 13 forks source link

LinformerEncoder Layer: No Linearisation? #3

Open JBKnights opened 3 years ago

JBKnights commented 3 years ago

Hello,

Sorry if this is a silly question, but looking at your code in ptr_base.py line 90 the LinformerEncoder layer doesn't seem to be implementing linear attention at all; what it seems to be doing instead is just performing regular multi-head attention. Is this the case, and if not where for the LinformerEncoder layers does the linearisation take place?

Thanks,

Josh

nmakes commented 2 years ago

Piggy-backing on @JBKnights question, I wonder if Linformer was intended.

From my understanding (please correct me if I'm wrong), the linearization in Linformer assumes an order in the inputs, e.g. features with (N, d) dimension are reduced to (K, d) where the N->K reduction will change if the order in the points (N,) changes. Since point clouds are sets of points and don't pertain to any ordering, the transformer layer should be order equivariant.

Thanks!