mathurinm / celer

Fast solver for L1-type problems: Lasso, sparse Logisitic regression, Group Lasso, weighted Lasso, Multitask Lasso, etc.
https://mathurinm.github.io/celer/
BSD 3-Clause "New" or "Revised" License
198 stars 32 forks source link

BUG - getting ``ConvergenceWarning`` despite convergence of the solver #248

Closed Badr-MOUFAD closed 2 years ago

Badr-MOUFAD commented 2 years ago
import numpy as np
from numpy.linalg import norm

from celer import celer_path
from celer.utils.testing import build_dataset

n_samples, n_features = 30, 50
X, y = build_dataset(n_samples, n_features)

alphas_max = norm(X.T@y, ord=np.inf) / n_samples
alphas, coefs, gaps = celer_path(X, y, 'lasso', max_iter=1, alphas=[
                                 alphas_max / 100.], verbose=2)
#########################
##### Computing alpha 1/1
#########################
Iter 0: primal 39.3909227664, gap 3.86e+01, 10 feats in subpb (50 left)
Epoch 10, primal 11.2687274551, gap: 3.82e-04
Epoch 20, primal 11.2687274532, gap: 2.60e-08
Exit epoch 20, gap: 2.60e-08 < 7.88e-05
c:\users\hp\desktop\celer-repo\celer\celer\homotopy.py:312: ConvergenceWarning: Objective did not converge: duality gap: 38.607043403374846, 
tolerance: 7.87818426033482e-05. Increasing `tol` may make the solver faster without affecting the results much.
Fitting data with very small alpha causes precision issues.
  sol = celer(
Badr-MOUFAD commented 2 years ago

Got it! Convergence of the inner problem doesn't imply convergence of the global problem.