search

jrbourbeau / dask-optuna

Scale Optuna with Dask
https://jrbourbeau.github.io/dask-optuna/
MIT License
35 stars 6 forks source link

Upstream? #19

Open mrocklin opened 3 years ago

mrocklin commented 3 years ago

This seems to have settled. Should we consider upstreaming this to Optuna proper?

jrbourbeau commented 3 years ago

Sure! Given Optuna's current joblib integration, this would just involve adding the DaskStorage (and corresponding OptunaSchedulerExtension) to Optuna. I'll inquire over in https://github.com/optuna/optuna/issues/1766 if the Optuna developers are interested in this.

There is a question as to whether or not Optuna will continue to support joblib (xref https://github.com/optuna/optuna/issues/1766#issuecomment-692616132) but, as was also pointed out in that issue (xref https://github.com/optuna/optuna/issues/1766#issuecomment-692899111), we could implement that functionality on the Dask side in a straightforward way.

jrbourbeau commented 3 years ago

Update: the Optuna developers are interested in adding DaskStorage to the Optuna codebase (xref https://github.com/optuna/optuna/issues/1766#issuecomment-725257649)