yanx27
commented
4 years ago @ShiQiu0419 Hi, thank you for your attention. You are right, the implementation of AS module in this code is slightly different from paper and it is actually a group self-attention. We find out that self- attention is better perform in semantic segmentation task when its as_neighbor is little. This change will also update in final paper of CVPR2020 :)
ShiQiu0419
Hi,
In the paper, equation 2 defines your proposal of relation function that takes dot-product as similarities, and f_i is point feature while f_k is group (neighbors) feature.
But in your codes (Line 140 of pointasnl_util.py) for that: weights = tf.matmul(transformed_new_point, transformed_feature1, transpose_b=True) # (batch_size, npoint, nsample, nsample)
transformed_new_point and tranformed_feature1 may not represent such point/group features since they both come from the same features: new_point. As far as I am concerned, point feature is from individual coordinates or aggregated from neighbors with a dimension of [N, C], and group feature is [N, K, C] corresponding to each point feature. It is more likely a self-attention of group feature rather than attention of point feature to group feature. Do I get misunderstanding here?
Rest of operations follow a manner of non-local self-attention with 1st half of transformed_feature (i.e. tranformed_feature1) computing similarites and 2nd half of transformed_feature (i.e. feature) regressing new_group_weights. And further update xyz and feature using the learned weights.
In general, AS module seems a non-local attention block for xyz and features.
Congratulations on your excellent work and thanks for sharing.