Quadratic optimizer with linear constraints

[charlesm93]

Summary:

Create a new function to perform quadratic optimization.

Description:

The goal is to maximize a function subject to a set of equality and inequality constraints. This is a generalization of the Lagrange optimization problem, sometimes termed the Karush–Kuhn–Tucker problem. The proposed feature solves this problem when the function we wish to optimize is quadratic. See http://discourse.mc-stan.org/t/quadratic-optimizier/4405 for more details.

The target function has the form: f(x) = x' H x + v x, where H is a matrix and v a vector. For starters, I want to implement the case where there is a linear constraint on x of the form Ax = b, and x > 0. We can then think of generalizations.

The call to the solver has the form a = quadratic_optimizer(H, v, A, b, theta, delta), where H, v, A, and b are all functions of parameters theta and data delta. Are there features in C++11 to make this easier to implement? Right now, my plan is to mimic what was done for the algebraic and the ODE solver.

Additional Information:

I can think of a few applications for this optimizer; the current motivating problem is in econometrics, brought forth Shosh Vasserman.

Current Version:

v2.17.0

[charlesm93]

One good candidate optimizer is the one built by eiquadrpogpp because it uses Eigen. Not sure what the license is here. See http://www.labri.fr/perso/guenneba/code/QuadProg/eiquadprog.hpp

[bob-carpenter]

It's an issue in that it complicates our licensing having more dependencies. Right now, we only depend on Boost, Eigen, and CVODES. But it's not a dealbreaker if there's functionality we need.

On Jun 5, 2018, at 9:32 PM, Charles Margossian notifications@github.com wrote:

One good candidate optimizer is the one built by cppipm because it uses Eigen. The license is MIT -- is that an issue?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[charlesm93]

This is something we'll eventually want to worry about. Right now, all first four arguments are functors which depend on theta and delta. These functors return a matrix or a vector. If theta contains var elements, than all four functors spit out matrices of var; when we run reverse-AD, we'll compute partial derivatives we do not need.

I think the best solution is to overload the function and allow users to pass a matrix or a vector of doubles to quadratic_optimizer instead of a functor.