Closed Jacob-Stevens-Haas closed 3 weeks ago
@himkwtn this is a decent issue that is a little bit mathy, but beyond the good first issue
tag, if you want to take it on
I found that StableLinearSR3 might be calculating the l2 regularization term incorrectly. https://github.com/dynamicslab/pysindy/blob/889ac6db3095923b01ebfb1070800141dddef9f4/pysindy/optimizers/stable_linear_sr3.py#L221
If I understand correctly, it should be self.threshold * cp.norm2(xi) ** 2
** 2 right?
as I understand it, this line is in a conditional for weighted l2 norm. So it should be something like
cp.norm2(np.sqrt(self.thresholds) * np.ravel(xi))
SR3
and its subclasses use weighted norms in two types of expressions; in both cases, it appears that those classes get it wrong. calculated using cvxpyThis is odd because the
xi
is a 1d vector, as isnp.ravel(...)
, which means both are probably computing the norm of a scalar, not a vector.calculated using
pysindy.utils.base.get_regularization()
This function doesn't have any unit tests, so it's tricky to see how it expects inputs, but this line indicates that the thresholds may not be properly applied (self.reg
is the lambda function returned byget_regularization
,coeff_full
is (n_features, n_targets) andself.thresholds
is (n_targets, n_features)). It seems like elementwise multiplication, rather than matrix multiplication, is what's desired here.Reproducing code example:
I haven't built a small regression problem to prove these things are wrong, because I haven't worked with
SR3
other than speeding up the tests in #393. But its easy to see thatget_regularization()
returns the norm of the inner product, even though weights are (I think?) supposed to be elementwiseError message:
Both show that the norm is 2.
Thoughts
Broadly speaking, here's what are probably the next steps. Caveat, very little of my work deals with
SR3
, and none withSR3
weighted thresholding, so I may get through step 1 at the most.pysindy.utils.base
thatSR3
uses.@briandesilva , @akaptano , @kpchamp, I know this is old code, but if you have any time in the coming months to review, I'd appreciate it.