search

py-why / causaltune

AutoML for causal inference.
Apache License 2.0
204 stars 29 forks source link

When calling fit() on an instance that was already fitted, resume the fit, instead of starting from scratch #53

Closed EgorKraevTransferwise closed 2 years ago

EgorKraevTransferwise commented 2 years ago

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like A clear and concise description of what you want to happen.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

EgorKraevTransferwise commented 2 years ago

https://github.com/microsoft/FLAML/issues/514

TimoFlesch commented 2 years ago

Hey Egor, would you like me to implement the feature as discussed in the ms issue above? Could set it up so that it resumes the fit automatically when .fit() is called a second/third/... time?

EgorKraevTransferwise commented 2 years ago

I think the right way to do it is to have a resume:bool=False extra argument to the fit() function, which if true makes us look at the results property that contains the results of previous runs, and convert it into correct values for points_to_evaluate and evaluated _rewards arguments, so the optimizer is aware of the results of previous runs. For simplicity, in this case we should discard any other user-supplied values for points_to_evaluate and evaluated _rewards arguments; This way, the new run could specify a different search space (eg add a couple of estimators) etc, but inherit the fit results from the prev run. After the resumed run is complete, we should also make sure that the scores property and all its children are correctly populated with the best result across both the resumed and the original run, see also the discussion in the FLAML issue above. Would you agree or would a different approach be better?

TimoFlesch commented 2 years ago

yup that makes perfect sense, I can probably give this a go on Monday :)