search

JuliaML / LossFunctions.jl

Julia package of loss functions for machine learning.
https://juliaml.github.io/LossFunctions.jl/stable
Other
148 stars 34 forks source link

Problem with weighted mean? #160

Closed ablaom closed 1 year ago

ablaom commented 1 year ago

Trying out 0.9.

What am I missing here?

julia> value(L1DistLoss(), [1, 1, 1], [2, 3, 5], AggMode.WeightedMean([4, 2, 1], normalize=false))
4.0

But the absolute value of the differences of prediction and ground truth are [1, 2, 4], so weighted mean = (4*1 + 2*2 + 1*4)/(4 + 2 + 1) = 12/7, no?

Also, I don't understand the normalize option? Isn't a weighted mean invariant to rescaling of the weights?

If I do change normalize=true I still don't get 12/7≈1.71:

julia> value(L1DistLoss(), [1, 1, 1], [2, 3, 5], AggMode.WeightedMean([4, 2, 1], normalize=true))
0.5714285714285714

which is 4/7, the mean of the weighted sum.

MilesCranmer commented 1 year ago

Seeing this too. Causing failure in this PR: https://github.com/MilesCranmer/SymbolicRegression.jl/pull/195

ablaom commented 1 year ago

@juliohm

juliohm commented 1 year ago

I think we should get rid of WeightedMean in future releases to avoid confusion. Most people are interested in WeightedSum. You can compare the two implementations here:

https://github.com/JuliaML/LossFunctions.jl/blob/e15657691c073798d7fa2509554797df567e6eea/src/losses.jl#L84-L116

juliohm commented 1 year ago

So basically, WeightedMean = WeightedSum / nobs, the same way Mean = Sum / nobs

juliohm commented 1 year ago

I will mark the issue as invalid, and we can work on a next breaking release that removes the AggMode submodule and also removes these specific cases with AggMode.WeightedMean and AggMode.None