Setting shrinking=True in CDClassifier with loss log and penalty l1 seems to not converge toward the optimal solution. Increasing the number of iteration does not change this.
It looks like some coordinates are screened out and never put in again?
Here is a script showing that setting shrinking=True does not converge:
import numpy as np
from lightning.classification import CDClassifier
n_samples = 100
n_features = 5000
rng = np.random.RandomState(42)
X = rng.randn(n_samples, n_features)
y = 2*(rng.randn(n_samples) > 0) - 1
lmbd = 0.1 * abs(X.T.dot(y)).max()
def loss(X, y, lmbd, beta):
y_X_beta = y * X.dot(beta.flatten())
return np.log(1 + np.exp(-y_X_beta)).sum() + lmbd * abs(beta).sum()
for shrinking in [True, False]:
clf = CDClassifier(
loss='log', penalty='l1', C=1, alpha=lmbd,
tol=0, permute=False, shrinking=shrinking,
max_iter=2000)
clf.fit(X, y)
print(f"Shrinking: {shrinking}; Loss = {loss(X, y, lmbd, clf.coef_)}")
The output:
Shrinking: True; Loss = 43.29080271832241
Shrinking: False; Loss = 42.165458588261096
Setting
shrinking=True
inCDClassifier
with losslog
and penaltyl1
seems to not converge toward the optimal solution. Increasing the number of iteration does not change this.It looks like some coordinates are screened out and never put in again?
Here is a script showing that setting
shrinking=True
does not converge:The output: