search

heal-research / HEAL.NonlinearRegression

Fit and evaluate nonlinear regression models.
Other
3 stars 0 forks source link

negative value in SSR_cond - SSR #1

Closed folivetti closed 1 year ago

folivetti commented 2 years ago

https://github.com/heal-research/HEAL.NonlinearRegression/blob/4ef8a4817f63f6e283ed1166c10a6fc2d03f780f/HEAL.NonlinearRegression/Statistics.cs#L263

is it possible to have a negative value when calculating SSR_cond - SSR? I'm thinking it is since we can actually escape a local optima and improve the results. In this case, should we change it to abs(SSR_cond - SSR)?

gkronber commented 2 years ago

This may happen if the NLSQ algorithm did not converge to a global optimum. The proposed change of taking the absolute would not produce correct results as we are using SSR in other places as well. In my opinion the correct solution is to raise an error.

Using a more robust optimization routine (e.g. CMAES and later fine-tuning using LM) should make the problem less likely.

Another option is to detect the error and then re-starting the while procedure with the newly detected optimum. This could be done whenever we detect a new best SSR in any of the t-profile calculations.

folivetti commented 2 years ago

I like the second option best. After I fixed the code this issue never happened again, so I guess this could happen in a very rare situation.

folivetti commented 2 years ago

actually it is still happening for the DemoExp benchmark, it seems to be a very tiny difference. Probably due to tolerance parameter in the optimization algorithm. I'll check it out later.

gkronber commented 1 year ago

I have refactored the t profile calculation and added restarting when a new optimum is detected.