The 'group' parameter cannot be applied in the roles of the fit_predict function in TabularAutoML. - Githubissues

sb-ai-lab / LightAutoML

Fast and customizable framework for automatic ML model creation (AutoML)

https://developers.sber.ru/portal/products/lightautoml

Apache License 2.0

1.08k stars 47 forks source link

The 'group' parameter cannot be applied in the roles of the fit_predict function in TabularAutoML. #153

Open yuuniee opened 2 weeks ago

yuuniee commented 2 weeks ago

🐛 Bug

The 'group' parameter cannot be applied in the roles of the fit_predict function in TabularAutoML.
As a result of backtracking the code, I found that the 'group' item was not included in the CV split process.

To Reproduce

Steps to reproduce the behavior:

Run -> TabularAutoML().fit_predict(..., roles={'target': TARGET_NAME, 'group': GROUP_NAME})
"lightautoml > reader > utils.py > set_sklearn_folds()" <- Here, check if the 'group: Optional[np.ndarray]' variable has a value.

Expected behavior

The CV split method changes depending on the 'group' designation above.

Additional context

This was resolved by adding the following code to line 322 of "lightautoml > reader > base.py".
"kwargs["group"] = train_data.loc[:, roles["group"]]"

The issue was found in version "0.3.8.1" and is unknown for other versions.

I am always grateful for your open source contributions. Thank you.

Checklist

[ ] bug description
[ ] steps to reproduce
[ ] expected behavior
[ ] code sample / screenshots

alexmryzhkov commented 2 weeks ago

Hi @yuuniee,

Thanks for the detailed description of the problem with the models validation. We will for sure take a look into it.

Alex