search

siboehm / lleaves

Compiler for LightGBM gradient-boosted trees, based on LLVM. Speeds up prediction by ≥10x.
https://lleaves.readthedocs.io/en/latest/
MIT License
333 stars 28 forks source link

Comparison with onnx-mlir #33

Closed jamespinkerton closed 1 year ago

jamespinkerton commented 1 year ago

Hi. I’m a big fan of the project.

I’m curious how the project benchmarks agains onnx-mlir. Is there a reason lleaves should be more performant?

Thanks so much!

siboehm commented 1 year ago

Doesn't look like they support it: see here and here. ONNX Runtime has support for it (as an interpreter obv), but their implementation is mostly optimized for single-batch.

The LightGBM -> ONNX -> Binary pipeline doesn't quite get you model parity, since the node_thresholds for the TreeEnsembleRegressor are float32, while LightGBM (and most other libs) use float64, which can (in theory) lead to arbitrary large divergences. Float32 is definitely nicer, reduces the bandwidth bottleneck a bit.

It'd be pretty fun to implement a TreeEnsembleRegressor lowering in onnx-mlir! Maybe I'll do that at some point. There are some fun passes to write, like optimizing LightGBMs bitvector-based categorical branching.