search

laughtervv / SGPN

SGPN:Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation, CVPR, 2018
MIT License
268 stars 62 forks source link

Some questions about loss calculation #9

Open LZDSJTU opened 6 years ago

LZDSJTU commented 6 years ago

Hello, I read the code on calculating the loss and I have some questions.

  1. line 167: group_mat_label = tf.matmul(pts_group_label,tf.transpose(pts_group_label, perm=[0, 2, 1])) I do not understand the meaning of this operation. Could you give an simple example for it.

  2. line 179~184: diffgroup_samesem_mat_label = tf.multiply(diffgroup_mat_label, samesem_mat_label) diffgroup_diffsem_mat_label = tf.multiply(diffgroup_mat_label, diffsem_mat_label)

    num_samegroup = tf.reduce_sum(samegroup_mat_label) num_diffgroup_samesem = tf.reduce_sum(diffgroup_samesem_mat_label) num_diffgroup_diffsem = tf.reduce_sum(diffgroup_diffsem_mat_label) I am not clear about these code.

  3. line 191: pos = tf.multiply(samegroup_mat_label, pred_simmat) I know that it want to compute the distance in the same group. But I do not know why this sentence can work. Could you provide an simple example or some equation?

  4. line 197: group_mask_weight = tf.matmul(group_mask, tf.transpose(group_mask, perm=[0, 2, 1])) I think that maybe this is the same opeartion as line 167, but I cannot undestand

  5. line 209~210: Pr_obj = tf.reduce_sum(pts_semseg_label,axis=2) Pr_obj = tf.cast(Pr_obj, tf.float32) I guess that all element in Pr_obj may be 1. So what the function of it as a weight?

LZDSJTU commented 6 years ago

Thank you very much. And you can choose any of them to answer.

BenjiDayan commented 6 years ago

Hi @LZDSJTU , which file are you looking at?

Edit: nvm it's model.py

BenjiDayan commented 6 years ago

@LZDSJTU the only one I've worked out so far since the code includes a comment:

  1. line 167: pts_group_label is a Batch x num_points x num_groups ndarray, I think one hot style so for each point if there are 50 groups then it has 49 0s and one 1. Transposed it's B x NG x NP, so multiplying with itself you get B x NP x NP. As the comment says # BxNxN: (i,j) if i and j in the same group So the B x NP x NP has 0s everywhere where point (i, j) not in same group, and 1s where they are.
densechen commented 5 years ago

Hi, @Ziggareto When I test the pre-trained model in area 1, I even got the 1 pre-trained model test in area 1 AP in 6 cate. What's more, this also happened in area 5. Do I set the wrong parameters? And Could you please provide the pergroupthres.txt file to us?

Instance Segmentation AP: [0.77272727 0.96409938 0.58364528 0.9 0.27223911 1. 0.77928571 0.67933635 0.70536131 0.48571429 0.59607843 0.8 0.32612594] Instance Segmentation mAP: 0.6818933128790516 Semantic Segmentation IoU: [0.98613754 0.98480023 0.93936964 0.95665216 0.82676315 0.95118879 0.9408741 0.89464145 0.88190051 0.95819404 0.83323726 0.91617618 0.88444496] Semantic Segmentation Acc: %f 0.973563616147

Looking forward your reply.

BenjiDayan commented 5 years ago

Hi @LittleLampChen I haven't been following for ages, but I think pergroupthres.txt is generated by valid.py? Have you run that already?

PatrickFeng commented 4 years ago

Issue 1: @Ziggareto has given the point.

Issue 2: diffgroup_samesem_mat_label reprensents points of same semantic class but not same instance. And diffgroup_diffsem_mat_label represents points of neither same class nor same instance.

Issue 3: pred_simmat represents ||Fsim1 - Fsim2||^2 in the paper. It is the calculation of Lsim.

Issue 4: Referd from the paper: "Since the semantic segmentation network is actually wrongly trying to bring pairs of points in our second similarity class closer together in feature space, we also add an α > 1 term to increase the weight of our loss to dominate the gradient from the semantic segmentation output branch."

Issue 5: When a point belongs to the background, the corresponding row of Pr_obj is 0 not 1. You can refer to section 3.1 of the paper