Understading the Penalty Object

[arose13]

I'm trying to implement a fairly complex custom penalty and I'm having a hard time understanding parts of the penalty object. From what I can understand it consists of 2 parts...

1. A regularization() function. Which appears to compute the actual cost for given coefficients coefs
2. A projection() function. Which (might?) be the gradient?

Is this true and if so could I or someone else make a documentation contribution showing how to implement a custom Penalty() class.

[arose13]

To clarify, this is because I'm trying to implement the Pliable Lasso in python and I'm trying to use your solvers. This model has 2 sets of coefficients that are being constrained, the coefs (beta (p)) and modifying coefs (theta (p x k)).

[vene]

Firstly, I should clarify that the the Penalty objects from penalty.py can only be used by the FISTA solvers. (In general it is quite difficult to have a generic mechanism for plug-and-play penalties, losses and solvers, because the solvers often need different APIs to interact with the problem.)

That said, the Penalty object might be a good way to implement Pliable Lasso with a FISTA solver for arbitrary losses. For a penalty function Ω(w):

• regularization(w) computes the value of the penalty term (i.e., the numerical value of Ω(w)).
• projection(w, alpha, L) computes the approximation using the proximal operator of function alpha*Ω (see section 2.3 of the FISTA paper, ie (i think) argmin_x (alpha*Ω(x) + L/2) || x - (w - (1/L) ∇[alpha*Ω(w)]) ||^2)

You can probably use the algo from the Pliable Lasso paper to compute the proximal operator. HTH