haifengl / smile

Statistical Machine Intelligence & Learning Engine
https://haifengl.github.io
Other
5.97k stars 1.13k forks source link

Possibly a bug in ElasticNet implementation #728

Closed buntec closed 1 year ago

buntec commented 1 year ago

Describe the bug In ElasticNet.java the raw response vector y is augmented with p zeros before calling LASSO.train, where it is centered. Consequently, the model is trained with a response vector not having p zeros at the end (instead it contains p times the negative of the mean of the augmented y). If I'm not mistaken, this means that the ridge penalty is not taken into account correctly, and that the intercept b is computed incorrectly.

This is just from looking at the code. I don't have numerical evidence to back this up. Apologies if I'm missing something here!

Expected behavior I think the response vector y should be centered before augmenting it with zeros. (This would probably require moving the centering of y out of LASSO.train.)

Actual behavior The model is trained with a response vector y not having p zeros at the end.

Code snippet N/A

Input data N/A

Additional context N/A

haifengl commented 1 year ago

Thanks for callout. I center first n elements of y2 now before calling LASSO. Please try master branch.