A CVXPY layer handling a quadratic problem can only deal with positive definite Q-matrices

[jackbaker1001]

Looking at: https://locuslab.github.io/2019-10-28-cvxpylayers/, scroll down to the heading "The OptNet QP".

You will see it is suggested that a QP is handled like:

n, m, p = 10, 5, 5
Q_sqrt = cp.Parameter((n, n))
q = cp.Parameter(n)
A = cp.Parameter((m, n))
b = cp.Parameter(m)
G = cp.Parameter((p, n))
h = cp.Parameter(p)
x = cp.Variable(n)
obj = cp.Minimize(0.5*cp.sum_squares(Q_sqrt*x) + q.T @ x)
cons = [A @ x == b, G @ x <= h]
prob = cp.Problem(obj, cons)
layer = CvxpyLayer(prob, parameters=[Q_sqrt, q, A, b, G, h], variables=[x])

In cvxpy, it is more natural to use the cp.quad_form() function. This, however, throws errors:

Q = cp.Parameter((n, n), PSD=True)
q = cp.Parameter(n)
A = cp.Parameter((m, n))
b = cp.Parameter(m)
G = cp.Parameter((p, n))
h = cp.Parameter(p)
x = cp.Variable(n)
obj = cp.Minimize(0.5*cp.quad_form(x, Q) + q.T * x)
cons = [A @ x == b, G @ x <= h]
prob = cp.Problem(obj, cons)
layer = CvxpyLayer(prob, parameters=[Q, q, A, b, G, h], variables=[x])

n, m, p = 10, 5, 5
mat = torch.randn(n, n, requires_grad=True)
Q = mat.T @ mat
qval = torch.randn(n, requires_grad=True)
Aval = torch.randn(m, n, requires_grad=True)
bval = torch.randn(m, requires_grad=True)
Gval = torch.randn(p, n, requires_grad=True)
hval = torch.randn(p, requires_grad=True)

layer = qp_layer(n, m, p)

y, = layer(Q, qval, Aval, bval, Gval, hval)

returns

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[33], line 11
      8 Gval = torch.randn(p, n, requires_grad=True)
      9 hval = torch.randn(p, requires_grad=True)
---> 11 layer = qp_layer(n, m, p)
     13 y, = layer(Q, qval, Aval, bval, Gval, hval)

Cell In[32], line 12, in qp_layer(n, m, p)
     10 cons = [A @ x == b, G @ x <= h]
     11 prob = cp.Problem(obj, cons)
---> 12 layer = CvxpyLayer(prob, parameters=[Q, q, A, b, G, h], variables=[x])
     13 return layer

File ~/opt/miniconda3/envs/convex_net/lib/python3.10/site-packages/cvxpylayers/torch/cvxpylayer.py:79, in CvxpyLayer.__init__(self, problem, parameters, variables, gp)
     77 else:
     78     if not problem.is_dcp(dpp=True):
---> 79         raise ValueError('Problem must be DPP.')
     81 if not set(problem.parameters()) == set(parameters):
     82     raise ValueError("The layer's parameters must exactly match "
     83                      "problem.parameters")

ValueError: Problem must be DPP.

Which is odd, because the problem does follow DPP. Indeed, if you run the problem through CVXPY (not CVXPY layers) using pure values for the Q-matrix (i.e, not a cp.Variable), it will solve without issue. I think the issue is related to using variables in the quad_form function.

This all causes a problem because using cp.sum_squares() only works if your Q-matrix is positive definite. I.e, we need to pass a square root. What about positive semidefinite matrices?

[brunompacheco]

In CVXPY, with Q being a parameter, the problem is DCP but not DPP.

Python 3.10.9 (main, Jan 11 2023, 15:21:40) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cvxpy as cp
>>> Q = cp.Parameter((2,2), PSD=True)
>>> x = cp.Parameter(2)
>>> cp.quad_form(x, Q).is_dpp()
False
>>> cp.quad_form(x, Q).is_dcp()
True

I believe this is because x.T @ (Qx) (the second product of expressions) doesn't comply with the ruleset.