search

mljar / mljar-supervised

Python package for AutoML on Tabular Data with Feature Engineering, Hyper-Parameters Tuning, Explanations and Automatic Documentation
https://mljar.com
MIT License
3.05k stars 406 forks source link

Error when using "mae" metric in Optuna mode #435

Closed Zacchaeus00 closed 3 years ago

Zacchaeus00 commented 3 years ago

Hi , I encountered the following error when training in optuna mode: Exception in LightgbmObjective The entry associated with the validation name "validation" and the metric name "mae" is not found in the evaluation result list [('validation', 'l1', 0.23193058898755722, False)]. Screen Shot 2021-07-16 at 17 57 19 Idk if it's a bug for the metric or I'm feeding bad data. Any help would be great. Thanks!

pplonski commented 3 years ago

@Zacchaeus14 looks like some metric bug - thank you for reporting. Do other metrics work without errors? If you remove LightGBM from algorithm list, then other algorithms work as expected for MAE metric?

Zacchaeus00 commented 3 years ago

@Zacchaeus14 looks like some metric bug - thank you for reporting. Do other metrics work without errors? If you remove LightGBM from algorithm list, then other algorithms work as expected for MAE metric?

Thank you for reply. The default rmse works. I haven't try removing LightGBM yet.

Zacchaeus00 commented 3 years ago

@pplonski I think the little bug is here: https://github.com/mljar/mljar-supervised/blob/200b4f36ae4da9df936522644028280b02bf767e/supervised/algorithms/lightgbm.py#L62 Hopefully, replacing "mae":"mae" with "mae":"l1" should work.

pplonski commented 3 years ago

Thank you @Zacchaeus14!