search

Evovest / EvoTrees.jl

Boosted trees in Julia
https://evovest.github.io/EvoTrees.jl/dev/
Apache License 2.0
175 stars 21 forks source link

Example for how to use EvoTrees.jl as a component of a more complicated model #256

Closed simsurace closed 12 months ago

simsurace commented 1 year ago

Hi, EvoTrees.jl is really great. I'd like to combine its predictions with another model and train the whole thing end-to-end. Is there an easy way to do this, perhaps using the MLJ.jl API?

jeremiedb commented 1 year ago

Thanks for the kind words! I'm not totally clear about the nature of the end-to-end training you want to implement, as EvoTrees isn't a differentiable algo, but for anything that concerns ensemble, stacking and the likes, MLJ may well be the way to go.

EvoTrees is well integrated with MLJ (essentially thanks to @ablaom direct involvement!). There's a tutorial showing a minimal usage with MLJ: https://evovest.github.io/EvoTrees.jl/dev/tutorials/examples-MLJ/ For more complex task, I'd refer to the MLJ doc https://github.com/alan-turing-institute/MLJ.jl as I don't have much experience on more sophisticated pipelines with MLJ. But don't hestate to reach out if you reach some blockers as I'm interest to better understand usage patterns and hopefully figure some improvements to the MLJ and related ecosystem.

simsurace commented 1 year ago

I think I confused myself there, thinking that there would be some loss through which to differentiate (even though the base learners are not differentiable), but I was mistaken. Thanks, those are good starting points. Maybe ensembling is all I need, so I'll start there.

jeremiedb commented 12 months ago

I'm closing as I think there wasn't specific issue or feature to be considered at the moment. Feel free to reopen a file a new one otherwise!