understanding of the loss

[pyedog1976]

Dear Author,

Thanks a lot for providing the code.

In your reco loss computation,

reco_loss = reco_loss + F.cross_entropy(seg_logits / temp, torch.zeros(num_queries).long().to(device))

I think the cross entropy part is equivalent with the nll loss and manually picks the positive index from the softmax outputs, as shown below:

-torch.log_softmax(seg_logits/temp, dim=1)[:, 0].mean()

However, I've found there always be slight different between those two computation ways, for example:

1). cross entropy way: 5.081362724304199
2). nll way: 5.081363677978516

My question is, do you think such different is caused by the floating point error? Am I misunderstand something?

Thanks.

Regards

[lorenmt]

Hello!

Yes, using log-softmax and cross-entropy should be mathematically the same. The difference here is very minimal and indeed can be caused by floating error, or implementation difference.

[pyedog1976]

Thanks for your reply, and I'm tending to say the floating error can be the reason.