The lava regularizer

[patwa67]

I'm trying to implement what is called the lava regularizer. Theoretically, it would look like:

@minimize ls(X*(b+c) - y) + lambdal*norm(b,1) + lambdar*norm(c,2)

However, I don't think this leads to an identifiable model. An alternative suggested by the authors (https://projecteuclid.org/euclid.aos/1487667617) is to first perform a Ridge solution, and then use coefficients and the penalty in a Lasso. However, I cannot figure out how to do the last step:

using StructuredOptimization

#Simulated data (n observations, p variables, tr true variables, sig error)
n, p, tr, sig = 500, 1000, 10, 0.1
X = randn(n, p)
b_true = [randn(tr)..., zeros(p-tr)...]
y = X*b_true+ sig*randn(n)

b = Variable(p);   # initialize ridge optimization variable
lambdar = 10.0
# Ridge
@minimize ls(X*b - y) + lambdar*norm(b,2)

bhat = copy(~b)
c = Variable(p);   # initialize lasso optimization variable
lambdal = 10.0
rpen = lambdar*norm(bhat,2) # Const Ridge penalty

#Lava (bhat and rpen are constants)
@minimize ls(X*(bhat+c) - y) + lambdal*norm(c,1) + rpen

[nantonel]

rpen is a constant: adding it to the cost function has no influence on the problem.

If you remove it from the last line:

@minimize ls(X*(bhat+c) - y) + lambdal*norm(c,1)

no error is raised and an equivalent optimization problem is solved.