JuliaStats / Lasso.jl

Lasso/Elastic Net linear and generalized linear models
Other
143 stars 31 forks source link

fit LassoPath currently not working? #11

Closed arturgower closed 6 years ago

arturgower commented 7 years ago

I'm using Julia 0.51 and Pkg.status("Lasso") = 0.1.0


y = x ->  1. + 0.*x + 3x^2 + 0.x^3 # choose true function

# training data
R = rand(20)*pi/2
Y = y.(R)

# select features
X = hcat(R.^0, R, R.^2, R.^3) # matrix with 4 columns

# L2 regression with no regularization works
X\Y # ~= [1.,0.,3.,0.]

# but Lasso does not
fit(LassoPath,X,Y)
> LassoPath (100 solutions for 4 predictors in 99 iterations):
         λ pct_dev ncoefs
  [1]  NaN     0.0      0
  [2]  NaN     NaN      4
  [3]  NaN     NaN      4
  [4]  NaN     NaN      4
  [5]  NaN     NaN      4
  [6]  NaN     NaN      4

# As λ = NaN, which is strange I tried specifying  
λs = reverse([1e-7,1e-6,1e-4, 1e-3,1e-2,1e-1,1])
fit(LassoPath,X,Y; λ=λs)
> LassoPath (7 solutions for 4 predictors in 7 iterations):
          λ pct_dev ncoefs
[1]     1.0     NaN      4
[2]     0.1     NaN      4
[3]    0.01     NaN      4
[4]   0.001     NaN      4
[5]  0.0001     NaN      4
[6]  1.0e-6     NaN      4
[7]  1.0e-7     NaN      4

....

Am I doing something wrong?
simonster commented 6 years ago

You have an intercept in your design matrix and Lasso.jl standardizes each column by its standard deviation by default, so your intercept turns into Infs. Depending on your goals, you can run fit as fit(LassoPath, X[:, 2:end], Y) (Lasso.jl fits an unpenalized intercept by default) or fit(LassoPath, X, Y; standardize=false, intercept=false).

arturgower commented 6 years ago

Thanks that's clear. I've added a warning explaining this and a mention of it in docs.