Calculation of the weights for the loss-function

[Sabze]

Hi, In your paper it says that the weights for the loss function (weighted cross-entropy loss) are calculated according to: w_c = 1/log(f_c + epsilon), where f_c is the inverse frequency of the class.

However, if I'm not mistaken, it looks like the weights are calculated differently in the code (in semantic/modules/trainer.py) w_c = 1/(content + epsilon), where content is the frequency of the class.

Am I misunderstanding something or are the weights calculated differently, and if that's the case, why are they calculated differently?

[tano297]

Hi, You are correct. The weight calculation is done differently than I stated in the paper in the latest version of the code, as I was trying different weighing strategies before I released the code. Thanks for pointing it out. Feel free to return it to the original version in your code. I will leave this open for other people to see as well

[Sabze]

Thanks for the quick reply, did you notice any difference when using different weighting strategies?

[Sabze]

I'm having some trouble understanding your original weight function (w_c = 1/log(f_c + epsilon)). Doesn't it give bigger weights for the classes with more points?

For example, if we have 75 points with label road, 20 points with label building and 5 points with label person, we get the frequencies [0.75,0.20, 0.05]. If I interpret the weight function correctly, then the weights for these labels would be: [3.47, 0.62, 0.33] (with epsilon = 0.001). Don't we want bigger weights for the classes with lower frequencies?

[tano297]

Hi,

Maybe epsilon is a confusing name for it, since epsilon is indeed usually a small number in the denominator. But in this case epsilon is higher than one, and here the whole story changes. Try with 1.02 for example :) In practice, you don't set the epsilon, but the maximum allowed weight, and calculate the epsion from this. I did not invent this, I borrowed it from enet

[Sabze]

That explains it, thank you! :)