sb-ai-lab / LightAutoML

Fast and customizable framework for automatic ML model creation (AutoML)
https://developers.sber.ru/portal/products/lightautoml
Apache License 2.0
1.09k stars 48 forks source link

Blending uses weights from last iteration, not best #67

Open dev-rinchin opened 1 year ago

dev-rinchin commented 1 year ago

🐛 Bug

Log from tutorial 1:

[10:39:41] Layer 1 training completed.

[10:39:41] Blending: optimization starts with equal weights and score 0.7529907972036357 [10:39:41] Blending: iteration 0: score = 0.7552430767490724, weights = [0.17297602 0.45194352 0.12763917 0. 0.24744129] [10:39:41] Blending: iteration 1: score = 0.7555030859886485, weights = [0.22631824 0.37608108 0.19821008 0. 0.1993906 ] [10:39:41] Blending: iteration 2: score = 0.7554997906957511, weights = [0.23017529 0.38376114 0.18327494 0. 0.20278871] [10:39:41] Blending: iteration 3: score = 0.755520519151073, weights = [0.23089828 0.38205293 0.18385063 0. 0.20319813] [10:39:41] Blending: iteration 4: score = 0.7555106332723811, weights = [0.22740835 0.3837866 0.18468489 0. 0.20412019] [10:39:41] Automl preset training completed in 234.02 seconds

[10:39:41] Model description: Final prediction for new objects (level 0) = 0.22741 (5 averaged models Lvl_0_Pipe_0_Mod_0_LinearL2) + 0.38379 (5 averaged models Lvl_0_Pipe_1_Mod_0_LightGBM) + 0.18468 (5 averaged models Lvl_0_Pipe_1_Mod_1_Tuned_LightGBM) + 0.20412 (5 averaged models Lvl_0_Pipe_1_Mod_3_Tuned_CatBoost)

CPU times: user 18min 28s, sys: 1min 26s, total: 19min 55s Wall time: 3min 54s

github-actions[bot] commented 1 year ago

Stale issue message