Doubt in vert_mask in run_skinning script

[dhorka]

Hi,

I have a couple of doubts related to how the loss is calculated. In specific in these lines:

   vert_mask = (torch.abs(skin_gt.sum(dim=1) - 1.0) < 1e-8).float()  #
   loss = cross_entropy_with_probs(skin_pred, skin_gt, reduction='none')
   loss = (loss * loss_mask_batch * vert_mask.unsqueeze(1)).sum() / (loss_mask_batch * vert_mask.unsqueeze(1)).sum()

I do not completly understand why is needed this vert_mask. The loss_mask_batch is the one used to filter the skinning_weights in the cases where k is greater than the number of bones of the current mesh, isn't it? But why is needed this vert_mask?

And the third line is not clear for me. I assume that you are trying to calculate the mean loss just for the real skinning weights, however, I do not see why you need to multiply for this vert_mask.

Thanks!

[zhan-xu]

From what I can remember, vert_mask filters out "bad skinning vertices" where their skinning weights are not summed to 1. Suppose you have perfect training data, this mask will be all 1, which means it is of no use. But in case where you have some noise data, some vertices may have all-zero skinning weights. We don't want to involve those vertices to affect training.

[dhorka]

Hi, sorry for my late response. I understand that you want that the sum of the skinning weights of a vertice should be 1 and for this reason you are using this mask. However, I found that in the current code, the prediction of the network is not always predicting skinning weights that are going to sum 1. In specific in this line of code. You are computing the softmax before filtering the skinning weights using the loss_mask. If I am not wrong, this is going to generate that the skinning weights of some nodes are not going to sum 1, isn't it?

[zhan-xu]

Yeah I think in https://github.com/zhan-xu/RigNet/blob/c5f5c985ca48700ac416b2f7f76a1fb3dde97554/run_skinning.py#L208, I normalized it again?

[dhorka]

Thanks! I have miss this line.