Open lzw19951010 opened 6 years ago
Hi,
I use tf.nn.sigmoid_cross_entropy_with_logits
for computing the loss while the original MatConvNet version use something like tf.nn.softmax_cross_entropy_with_logits
to compute the loss.
For tf.nn.sigmoid_cross_entropy_with_logits
, the label is the probability of the position being the target, therefore, 0 or 1 or 0.5 for not sure. As for tf.nn.softmax_cross_entropy_with_logits
, the label is the category of the position belonging to, therefore +1 for target, -1 for background and 0 for not sure.
Please convince yourself these two implementations are identical.
Cool, I did not take the loss function into account. It seems to me that these two implementation are optimizing the score response towards similar targets. However, I still cannot convince myself they are exactly identical. Because in softmax_cross_entropy, score inter-class are non-independent while they are independent in sigmoid. Probably a solid conclusion can be only derived from their backward propagation formulas.
Thank you for your code and your patience!
Hi, bilylee, thank you for your great work!
I've noticed a slight difference between your implementation and Luca's. I'm not sure whether it is reasonable, it would be great if you can enlighten me on this.
Luca's implementation:
Yours:
Both of which used balanced weights, but you only generated 1/0.5/0 with your default settings, while Luca's implementation generated 1/0/-1 with respect to different radius. I'm not sure whether this is different by design or simply I'm wrong about it.