Open trivialfis opened 1 year ago
Hi, great work on the initial multitarget implementation!
Given the roadmap when can we expect GPU support for multi output regression? When this support is added will xgboost-ray
also support it?
Hi @CarloLepelaars ,
Hi, very nice work! I am wondering how SHAP should be used for multi-output models, e.g. how to explain links between the Ys, and how to interpret the effects of Xs - e.g., which Xs display common effects across the Ys, and which Xs display differential effects. Do you know a good example of using SHAP for a multi-output model?
For model per target, it's the same as single target. As for vector leaf, I haven't looked into it yet, but no significant difference on top of my mind.
I am currently toying with multitargets approach ... I have a hard time defining a custom metric (haven't tried custom loss). Preds seems to be of size (len(y) x len(targets)) while y_true is of shape (len(y), len(targets)), I have managed to handle this internally to my metric to return one value. But now I have an error about an output being a tuple instead of a number. Any way to handle this properly or is it too early ?
Hi.
Did anybody train the multiple outputs XGBoost model on Mac arm64 machine?
On recent stable version I have got error:
XGBoostError('[...] Check failed: !trees.front()->IsMultiTarget(): Update tree leaf support for multi-target tree is not yet implemented.
On latest nightly version xgboost-2.1.0.dev0+a7226c02223246be78a59c3a4e8c32d1c68c1ff9 - I have managed load CPU, but it was no feedback on terminal window.
Is the vector-leaf-based multi-output model still work in progress ? Also what research paper based on which splitting mechanism for decision trees is working for this ? @trivialfis
yes, it's still working in progress.
Hi @trivialfis,
I'm currently working on some models using XGBoostLSS which as far as I understand is based on the multi-output feature of XGBoost. I wonder how monotonic constraints are considered in the multi-ouput case ? It seems constraints are shared among trees built for each target, could you confirm ?
Thanks for your work on this feature !
Hello @trivialfis, I'm working on a multi label binary classification problem (I have three targets) and all my targets are highly imbalanced. But I don't seem to understand how I can leverage scale_pos_weight
to help with that.
Also, will I be able to access functionalities such as shapley values computation?
Since the XGBoost 1.6, we have been working on having multi-output support for the tree model. In 2.0, we will have the initial implementation for the vector-leaf-based multi-output model. This issue is a tracker for future development and for related discussion. The original feature request is here: https://github.com/dmlc/xgboost/issues/2087 . The related features are for vector leaf instead of general multi-output.
Feel free to share your suggestions or make related feature requests in the comments.
Implementation Optimization
Algorithmic Optimization
We are still looking for potential algorithmic optimization for vector-leaf and here's the pool of candidates. We need to survey all available options. Feel free to share if you have ideas or paper recommendations.
GPU Implementation
Documentation
Multi-task
Features
Learning to rank
We can have a ranking model to consider multiple criteria. This might require multi-task to be supported.
Quantile regression
Distributed
Binding
HPO
Other extensions
Applications
Benchmarks