Closed roborocklsm closed 3 years ago
Yes, there should be a 1/2 in softmax according to the derivation. But in practice, we find this constant factor hardly influence the final performance, so we remove the 1/2.
@Zhongdao Yes, sure, you are right. I have tried and it does have no difference between two results.
Thx.
Hey, thanks for your great work! But I have a question about the Automatic Loss Balancing Function.
According to Kendall, Gal, andCipolla 2018, continuous output and softmax output have different coefficients: continuous one contains 1/2 while softmax does not. But I notice both in your paper and code give coefficient 1/2 to all three losses.
Would you mind giving some further explanations?