Closed encore-zhou closed 4 years ago
I just checked. I agree with you. Let me change the code and test it. Thank you for pointing this out!
Thank you for your suggestion! We have fixed the issue, and it offers a marginal improvement on ScanNet, and does nothing on SUNRGB-D. We have updated the code. Thank you, again!
Hi, Thanks for your reply. I have other questions as below.
The outputs of PrimitiveModule consists of three parts(offset, semantic scores and size residuals). The offset and semantic is well explained in the paper. But I cannot find what "size residuals" means. And I found that the size residuals of different primitives are not used during inference time.
For example, in file proposal_module_surface.py
: Line 28:
def decode_scores(net, end_points, num_class, mode=''):
net_transposed = net.transpose(2,1) # (batch_size, 1024, ..)
batch_size = net_transposed.shape[0]
num_proposal = net_transposed.shape[1]
base_xyz = end_points['aggregated_vote_xyz'+mode] # (batch_size, num_proposal, 3)
center = base_xyz + net_transposed[:,:,0:3] # (batch_size, num_proposal, 3)
end_points['center'+mode] = center
if mode == '_z':
end_points['size_residuals'+mode] = net_transposed[:,:,3:5]
sem_cls_scores = net_transposed[:,:,5:] # Bxnum_proposalx10
end_points['sem_cls_scores'+mode] = sem_cls_scores
elif mode == '_xy':
end_points['size_residuals'+mode] = net_transposed[:,:,3:4]
sem_cls_scores = net_transposed[:,:,4:] # Bxnum_proposalx10
end_points['sem_cls_scores'+mode] = sem_cls_scores
else:
sem_cls_scores = net_transposed[:,:,3:] # Bxnum_proposalx10
end_points['sem_cls_scores'+mode] = sem_cls_scores
return center, end_points
Size residuals are decoded from the outputs of `PrimitiveModule`.
Thank You!
For question 1, when I fixed the bug, it just gives me marginal improvements, which mean small incremental improvements, in ScanNet.
For question 2, Yes, Size residuals means the length of an edge or size of a surface. It's not used in inference. For now, we just use it as a additional supervision and hope it can offer some improvements through multi-task. However, if you remove it, I doubt that it will have any impact. It's not an important factor.
Btw, I was just curious and I looked at your github. I found that you were trying to add the primitive part to Open-mmlab, right? Did you get any improvements by adding the primitive part? I am just curious. If it's confidential, feel free to not share it.
Hi, I find that there may be some mistakes in
PointnetSAModuleMatch
module.In file proposal_module_refine.py: line 328:
surface_xyz, surface_features, _ = self.match_surface_center(torch.cat((obj_surface_center, surface_center_pred), dim=1), torch.cat((obj_surface_feature, surface_center_feature_pred), dim=2))
In line 119:
match_surface_center
is defined as:As for
torch.cat((obj_surface_center, surface_center_pred), dim=1)
andtorch.cat((obj_surface_feature, surface_center_feature_pred), dim=2)
, we can get a set of points and their correspondence features.Then, I go further into the source code of
PointnetSAModuleMatch
module: pointnet2_modules.py: Line 467:Here, I think the difference between
PointnetSAModuleMatch
and originalPointnetSAModule
is that user can specify new_xyz inPointnetSAModuleMatch
module. However, I found some problems in Line 472.target_xyz
andfeatures
are two parameters parsed to functionself.grouper
, but they are mismatch in these case. Problem will happen while grouping the neighbour features, because the features oftarget_xyz
is begin fromfeatures[:,:,self.npoint:]
. I think we should correcte this code with