search

liuzhengzhe / One-Thing-One-Click

https://arxiv.org/abs/2104.02246 One Thing One Click (CVPR 2021) https://arxiv.org/abs/2303.14727 One Thing One Click++ (Arxiv)
53 stars 8 forks source link

Questions in merge part #14

Open argosdh opened 2 years ago

argosdh commented 2 years ago

Hi, I have several questions in merge section. From my understanding, the merge part does the graph propagation based on results of 3D-Unet and RelationNet. However, in ./merge/model/pointgroup/pointgroup.py, only U and U_prod were used to generate avg and conf. Other variables like result_avg_crf and result_avg were never used. And I'm also confused about the algorithm of graph propagation. In ./merge/test_train.py, Q[i] means the ith super voxel's class semantic score calculated by production of super voxel pooled 3D-Unet semantic prediction U(M, 20) and relation net prediction of each labelled super voxel U_prod(M, 20). In U_prod, only those super voxels with labels are scored, other lines are all zeros. Why you use softmax(U U_prod 5) as the final class prediction of each super voxel, I can't connect this line with the paper. Also, the confidence(conf) is calculated by torch.max(torch.cat(U,U_prod)). Are they comparable? Just choose the biggest score out of 40 scores and why this largest score is the one which means to be the confidence of super voxel SV belongs to class C?

liuzhengzhe commented 2 years ago

Thanks for your message.

  1. Other variables like result_avg_crf and result_avg: I used them to check intermediate results and see how these operations influence the performance.
  2. softmax(U U_prod 5) as the final class prediction: We have mentioned it in paper page 5 "The prediction of relation network is further combined with the prediction of 3D U-Net by multiplying the predicted possibilities of each category to boost the performance"
  3. the confidence(conf) is calculated by torch.max(torch.cat(U,U_prod)): we use the simple thresholding way to update pseudo label, and the maximum value is the prediction confidence.
derkaczda commented 2 years ago

About 2.: Why multiply by 5, this seems like a random number to me. Similar to that why do this loop 5 times. https://github.com/liuzhengzhe/One-Thing-One-Click/blob/c6c2e05b57663c4fa490195d09c547b6fd197821/merge/model/pointgroup/pointgroup.py#L392-L395

liuzhengzhe commented 2 years ago

It is a hyper parameter, and can be set to other values.

derkaczda @.***> 于2022年2月10日周四 00:13写道：

About 2.: Why multiply by 5, this seems like a random number to me

— Reply to this email directly, view it on GitHub https://github.com/liuzhengzhe/One-Thing-One-Click/issues/14#issuecomment-1033933675, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB7WPDFO2TAKUTVPQ6ZI3QDU2KHEHANCNFSM5IEIMWSA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

c6376315qqso commented 2 years ago

Thanks for your message.

  1. Other variables like result_avg_crf and result_avg: I used them to check intermediate results and see how these operations influence the performance.
  2. softmax(U U_prod 5) as the final class prediction: We have mentioned it in paper page 5 "The prediction of relation network is further combined with the prediction of 3D U-Net by multiplying the predicted possibilities of each category to boost the performance"
  3. the confidence(conf) is calculated by torch.max(torch.cat(U,U_prod)): we use the simple thresholding way to update pseudo label, and the maximum value is the prediction confidence.

Hi, I can't see where the crf is in the code, it seems like in the code you didn't use a crf to predict labels?

liuzhengzhe commented 2 years ago

We implement CRF using pytorch. Please check merge/model/pointgroup/pointgroup.py: line 383 to 402.