tBuLi / symfit

Symbolic Fitting; fitting as it should be.
http://symfit.readthedocs.org
MIT License
233 stars 17 forks source link

Constraints with any minimizer using Lagrange multipliers #148

Open tBuLi opened 6 years ago

tBuLi commented 6 years ago

This feature is long overdue, see also #31. What I would like to do as a first step towards this is to build a simple support function that when given a model and constraints, outputs the Lagrange function to be optimized. Example:

from symfit import parameters, Eq
from symfit.core.support import lagrange_function
x, y = parameters('x, y')

f = x + y
unit_circle = Eq(x**2 + y**2, 1)
model = lagrange_function(f, constraints=[unit_circle])
print(model)
>>> x + y + l1 * (x**2 + y**2 - 1)

This is a very minimal example, in case you don't manually provide parameters to be used as Lagrange multipliers. When you do, feed the constraints argument with a dict instead:

from symfit import parameters, Eq
from symfit.core.support import lagrange_function
l, = parameters('l')
model = lagrange_function(f, constraints={l: unit_circle})
print(model)
>>> x + y + l * (x**2 + y**2 - 1)

This is recommended, since the automatic generation might result in conflicts.

I'm not sure yet if this function should return a Model object, or simply an expression that you yourself can then feed to a model. Probably the latter, since that way you can name the dependent variable yourself.

This model can then be fed to any existing minimizer and it will perform a constraint minimization instead.

Additionally, we can use this same syntax to help Fit distinguish between these approaches as well. If you feed Fit the constraints as a list, it will continue to use SLSQP. If you feed it a dict with parameters as keys instead, it will call this function first and minimize the Lagrange function using whichever minimizer it deems best based on the usual criterions.

fit = Fit(f, constraints={l: unit_circle})
fit_result = fit.execute()

This feature can also be extended to include Karush–Kuhn–Tucker inequality constraints.