Closed wuchihsu closed 2 years ago
Currently, the splitter is decided by the split_type
:
https://github.com/microsoft/FLAML/blob/a99e939404caeda88f32724cc264841f2f5dcfca/flaml/automl.py#L812-L853
Is your use case beyond one of them?
Yes, my case is beyond them. I want to set a customized splitter. Just like the parameter 'cv' in sklearn.model_selection.RandomizedSearchCV.
Not supported currently. Would you like to work on adding that feature?
OK, I would give it a try.
OK, I would give it a try.
Great. Feel free to discuss on gitter if you have questions.
@wuchihsu what's the status of this feature? #320 requests the same feature.
@wuchihsu what's the status of this feature? #320 requests the same feature.
Sorry, I have not started yet. I can start this month.
Hi, thank you for your answer. I've tried to take a look myself at the code, but it seems like that such a modification would require refactors at multiple levels. If you can provide a high-level overview, I might be of some help as well.
Thanks! We would love to add support for customized splitters. We envision the API for a customized splitter to be like a class with an implemented function called split(X_train), which will be called here https://github.com/microsoft/FLAML/blob/dd60dbc5eb7e6d298c5f1506a8e0b7a8bb390052/flaml/ml.py#L323. Do you think this API would fulfill your need or in your split() function, you will need information more than X_train, e.g., y_train or group information? It would be even better if you could share your test code/data with us.
Do you want to work on adding this feature? At a high-level, there are a few places that need to get changed:
Please let me know your thoughts. thanks.
I have made a work version -> wuchihsu/FLAML@a89f8837835c64f9be3cc6c28b6b0d6209881d9e. But I use a new param called custom_split
, because I haven't seen @slhuang reply when I began to work. The custom_split can be Iterable
or a class with split
and get_n_splits
method. Any suggestions? I can also change it to only use the param split_type
.
@wuchihsu Can you use the param split_type
and make a PR? We can then review your code. thanks!
@wuchihsu Can you use the param
split_type
and make a PR? We can then review your code. thanks!
I made a PR #333
Is it possible to set a splitter or a iterable when fitting?