请教一些问题

[Jerryme-xxm]

作者您好，我最近在尝试对您的PointGnn做一些改动。但是由于我的水平有限，不知道这些改动是否可行，有一些问题想要问您： 1、为什么您选用体素采样这种计算量较大的采样方法，是否有其他方法可以替代，如FPS最远点采样？ 2、如果对图顶点的邻点按照该点密度/距离加入权重，是否会对训练结果产生好的影响。比如：顶点为P，邻点A有3个采样点，与P点的距离为5；邻点B有5个采样点，与P点的距离为3，是否可以对B点加入一个比A点更高的权重？ 3、这种网络利用局部特征进行检测，是否可以加入全局特征来提高准确率？考虑到这必然会加大计算量，最后考虑这个改动方向。

[WeijingShi]

Hi @Jerryme-xxm, Those are excellent questions.

1. Why use voxel downsample instead of FPS? We want the downsampled point cloud to keep the general shapes. Voxel downsampling is pretty common in the point cloud libraries and it has decent speed. We tried FPS but our FPS implementation is quite slow so we did not use it. Maybe a better FPS implementation can improve both accuracy and speed. It sounds to be an interesting topic.

2. I think it's possible A is still more important than B, for example, all B's sampling points are the same, but A has very critical sampling points. Maybe it's better to use an attention mechanism and let the network itself figure out the right weights.

3. I think we can build a graph feature pyramid to combine different feature scales. You are correct it will increase the cost.

Hope it helps!

[Jerryme-xxm]

1，Hello, I've written a simple attention mechanism that mimics gacnet, as follows:

def attget(input_vertex_features, input_vertex_coordinates, edges, k = None):

Gather the source vertex of the edges

s_vertex_features = tf.gather(input_vertex_features, edges[:, 0])
s_vertex_coordinates = tf.gather(input_vertex_coordinates, edges[:, 0])
print("s_vertex_features",s_vertex_features)
print("s_vertex_coordinates",s_vertex_coordinates)
# Gather the destination vertex of the edges
d_vertex_features = tf.gather(input_vertex_features, edges[:, 1])
d_vertex_coordinates = tf.gather(input_vertex_coordinates, edges[:, 1])
# concat coordinates and features
p_h = tf.concat(
    [s_vertex_coordinates - d_vertex_coordinates, s_vertex_features - d_vertex_features],
    axis=-1)
alpha = slim.fully_connected(p_h, 300,
    activation_fn=partial(tf.nn.leaky_relu, alpha=0.01),
    normalizer_fn=slim.batch_norm
    )
# alpha = tf.nn.dropout(alpha, 0.5)
alpha = tf.nn.softmax(alpha, axis=-1)
return alpha

Then I add it to gnn.py as follows:

    with tf.variable_scope('extract_vertex_features'):
        # Extract edge features
        edge_features = self._edge_feature_fn(
            edge_features,
            Ks=edge_MLP_depth_list,
            is_logits=False,
            normalization_type=edge_MLP_normalization_type,
            activation_type=edge_MLP_activation_type)
        alpha = attget(input_vertex_features, input_vertex_coordinates, edges)
        edge_features = tf.multiply(alpha, edge_features)
        # edge_features = tf.contrib.layers.bias_add(edge_features)

Is there anything wrong in my code? My pc can run your original code with bs = 1, and when i add my attention mechanism to it, I don't have enough GPU(1660ti @6G) resources. Thus is there ang method to help?

2，In GACnet and GAT, the feature dims both reduce to lower dims, so their attention mechanisms have a dimension reduction process, but point-gnn`s feature dim doesn't reduce.Do you have any advice? Looking forward to your reply.

[WeijingShi]

Hi @Jerryme-xxm,

1. In gacnet paper equation 3, the attention is normalized for all edges connecting to the same vertex. In your implementation, the softmax is on the feature dimension instead. I think you may want something like:

def segment_softmax(alpha, edges, num_vertices):
    # Numeric stability 
    max_alpha_per_vertex = tf.math.unsorted_segment_max(alpha,
        edges[:, 1], num_vertices)
    alpha = alpha - tf.gather(max_alpha_per_vertex, edges[:, 1])

    # take exp
    exp_alpha = tf.exp(alpha)

    # sumerize all weights to the same vertex 
    sum_exp_alpha = tf.math.unsorted_segment_sum(exp_alpha,
        edges[:, 1], num_vertices)

    # normalize
    normalized_alpha =  exp_alpha / tf.gather(sum_exp_alpha, edge[:, 1])
    return normalized_alpha

So your attget code use this function instead of softmax:

normalized_alpha = segment_softmax(alpha, edges, num_vertices)

I did not test the code. Please let me know if I misunderstand your intention or the paper.
Also, the aggregation function of gacnet seems to be sum instead of max. You may want to use graph_scatter_sum_fn to aggregate the edges features later instead of the default graph_scatter_max_fn that I used.

As for the memory limitation, you can try to cut the graph into smaller pieces and process them one by one. But one 1660ti is indeed limiting.

2. If we want the output feature dims to be different with the input features dims, we can simply remove the residue connection here: https://github.com/WeijingShi/Point-GNN/blob/48f3d79d5b101d3a4b8439ba74c92fcad4f7cab0/models/gnn.py#L372 This would no longer constrain the output dimension.

Hope it helps. Thanks, Weijing

[Jerryme-xxm]

Thanks for your kindly help sincerely! These days I used one 2080 GPU successfully to train the network. I get my attention weight only through feature difference, because I think edge feature has used coordinate difference already. I only use one full_connect to extract attention weight, and only add it to layer 3. Finally, the precision increase by about 1, at the same time, time cost increase by 0.1.

[phoebedddd]

2080 gpu来了！！热心帮助！ 这几天天天天天成功个个个一了一一一一一一一一一一了了了一了来来来来来来来来来来训练。网络。。。。 我我只只只通过通过特征特征获得来差异差异差异来attention weight，只加到layer 3。 最后precision增加了1左右，同时time cost增加了0.1。

Hello, may I ask how you managed to train the network with only a piece of 2080? I really need your help. Could you please tell me?

[Jerryme-xxm]

2080 gpu来了！！热心帮助！ 这几天天天天天成功个个个一了一一一一一一一一一一了了了一了来来来来来来来来来来训练。网络。。。。 我我只只只通过通过特征特征获得来差异差异差异来attention weight，只加到layer 3。 最后precision增加了1左右，同时time cost增加了0.1。

Hello, may I ask how you managed to train the network with only a piece of 2080? I really need your help. Could you please tell me?

Sorry, I don't remember very clearly. At that time, I seemed to set the batchsize to 1 and complete the training in two days.