Minimize the sum of squares of terms with different magnitude

[gurtajbir]

Hi. I am trying to minimize a sum of three terms for a parameter estimation problem where the magnitude of one of the terms is larger ( about an order of magnitude) than the other two. When I perform the optimization, it results in a much better fit on the parts of the system that affect the term with the larger magnitude. Could you please suggest how this can be avoided ?

[gurtajbir]

More concretely, I have set up a UDE for estimating the missing physics for a system with 3 states similar to this example

Here is my loss function

`function loss(θₚ)

mse = 0
window = 30 
t_sample = [150, 300] 
for i in t_sample
    X̂ = predict(θₚ, ground_truth[:, i], t[i:i+window])
    e1 = mean(abs2, (ground_truth[1, i:i+window] .- X̂[1, :]))
    e2 = mean(abs2, (ground_truth[2, i:i+window] .- X̂[2, :]))
    e3 = mean(abs2, (ground_truth[3, i:i+window] .- X̂[3, :]))
    mse += e1 + e2 +e3
end
return mse

end `

[Vaibhavdixit02]

Edit - I misunderstood the issue before

You could give some weights to the errors and run the optimization.

a simple example assuming e3 ~ 1e6 e1, 1e6 e2 -

mse += e1 + e2 + (e3 * 1e-6)

[gurtajbir]

Thank you for the suggestion, @Vaibhavdixit02. I tried something similar where I was trying to weigh the terms by using a factor that inflated the contribution of e1 and e2 where they are defined. This seemed to work OK. I'll try out your suggestion as well and get back.