[RFC] [python-package] remove h2o `datatable` support?

jameslamb commented 2 weeks ago

Summary

Support for the h2o's datatable library was added to LightGBM 5.5+ years ago, in #1970.

Proposing here that lightgbm:

issue a deprecation warning for the next 2-3 releases whenever datatable is used
permanently remove datatable support 2-3 releases from now

Motivation

That project seems to be abandoned:

last commit and PyPI release was 10 months ago:
- https://github.com/h2oai/datatable
- https://pypi.org/project/datatable/
seems that h2o has moved on to something new called H2OFrame in the h20-py package:
- https://docs.h2o.ai/h2o/latest-stable/h2o-py/docs/frame.html#h2oframe
- https://github.com/h2oai/h2o-3/blob/6191acc115df654c5964225480d7d4542c22bed0/h2o-py/h2o/frame.py#L39

In those 5.5 years since #1970, the only bug reports / feature requests received about datatable support have been from one person working for h2o... and the last of those was 4 years ago:

(Sep 2020) #3386
(Feb 2019) #2003
(Jan 2019) #1968

And in all that time, I don't think we have ever tested against datatable in CI.

Description

Doing this would simplify the Python package, making it easier for others to contribute.

It'd also make it more manageable to add support for newer, more popular input formats like polars (#6204).

See @trivialfis's summary of the current state of supporting data frame libraries at https://github.com/dmlc/xgboost/issues/10554#issuecomment-2211824457 ... I agree with it.

References

I am not proposing here that lightgbm should support H2OFrame... Dask doesn't, XGBoost doesn't, scikit-learn doesn't... and I think our limited time and attention here would be better spent on more widely-used input formats, like polars.

jameslamb commented 2 weeks ago

@guolinke @shiyu1994 @StrikerRUS @jmoralez @borchero @btrotta please let me know what you think whenever you have time