search

sberbank-ai-lab / LightAutoML

LAMA - automatic model creation framework
Apache License 2.0
887 stars 92 forks source link

Reinitialize linear models #81

Closed PhySci closed 2 years ago

PhySci commented 2 years ago

It turned out that linear models are initialized only once when corresponding TorchBasedLinear model (LinearRegression or LogisticRegression) are being created. And consequent optimization procedures with different regularization coefficients do not reinitialize it. Therefore, all optimization procedures start not from scratch, but from the last state of the previous optimization. It means that optimizations are not fare and their comparison is incorrect.

In this PR, reinitialization of linear model before each optimization procedure has been implemented.

alexmryzhkov commented 2 years ago

As we discussed, this non-reinitialization is just a specific way for the warm start and speedup the models.