Bounds can cause fitting to fail

[tBuLi]

In some cases, supplying symfit with bounds to parameters makes the fitting converge to a non-solution, whereas without it finds the solution just fine. Since all bounds are handled by leastsqbound I think the bug is in that program.

It would therefore be prudent to reintroduce the Minimize facilities and use them to handle all fitting.

[tBuLi]

As an update, Minimize has now been reintroduced, wrapping scipy.optimize.minimize. This can now be used to take over fitting in the near feature, although the option for MINPACK should still be there given it's popularity. Potentially use MINPACK when no bounds are given, and Minimize when there are.

[pckroon]

Referencing #67, #86, #75.

[Jhsmit]

I'm setting up my model in the following way (python 2.7):

a1 = Parameter(name='a1', value=200, min=0, max=1000)
a2 = Parameter(name='a2', value=300, min=0, max=2000)
a3 = Parameter(name='a3', value=400, min=0, max=2000)

b = Parameter(name='b', value=200, min=0, max=500)

t1 = Parameter(name='t1', value=0.0001, min=0)
t2 = Parameter(name='t2', value=0.07, min=0)
t3 = Parameter(name='t3', value=1., min=0)

t = Variable(name='t')

model = a1*exp(-t/t1) + a2*exp(-t/t2) + a3*exp(-t/t3) + b

This gives some warnings:

C:\Python27\lib\site-packages\symfit\core\fit.py:45: RuntimeWarning: invalid value encountered in sqrt
  stdevs = np.sqrt(np.diagonal(pcov))

I didnt get the warning in symfit 0.2.5, i do in 0.3.5. The warning goes away when adding upper bounds to the tx parameters. Is it no longer allowed to specify only left bound?

[tBuLi]

Could you try to run your fitting with the NumericalLeastSquares and ConstrainedNumericalLeastSquares objects directly? My prediction: you have this problem only with the latter, which is used by Fit by default for bounded problems since 0.3.5.

If true, could you explicitely set your upper bound to None for ConstrainedNumericalLeastSquares ?

t1 = Parameter(name='t1', value=0.0001, min=0, max=None)

I guess there might be a difference in syntax between the two wrapped scipy methods.

[Jhsmit]

Your prediction is correcct, when I use NumericalLeastSquares directly I don't get warnings or error message in the fit result report. I do get a warning when I sueConstraintNumericalLeastSquares;

C:\Python27\lib\site-packages\scipy\optimize\slsqp.py:338: RuntimeWarning: invalid value encountered in greater
  bnderr = where(bnds[:, 0] > bnds[:, 1])[0]

Fitting error: Fitting status message: Inequality constraints incompatible. This is a different error than I originally reported. I might have changed something, cant find what it is. Setting max=None doenst fix anything.

When I configure the parameters in the following way:

a1 = Parameter(name='a1', value=200, min=0, max=1000)
a2 = Parameter(name='a2', value=300, min=0, max=2000)
a3 = Parameter(name='a3', value=400, min=0, max=2000)

b = Parameter(name='b', value=200, min=0, max=500)

t1 = Parameter(name='t1', value=0.0001, min=0, max=1000)
t2 = Parameter(name='t2', value=0.07, min=0, max=1000)
t3 = Parameter(name='t3', value=1., min=0, max=100)

I dont get the warning, but I do get Fitting status message: Inequality constraints incompatible. Either I'm stupid and I've chose the guess outside of the bounds (but I dont see where) or its not related to only specifing left bound. No bounds = No problems. The fitting acutally works quite well even with mediocre guesses, and works but slow with poor guesses.