jameslamb
commented
1 year ago Thanks for using LightGBM.
My best guess is that there is some sorting by covariates happening at each boosting stage that doesn't apply to my hackily tacked on dataset attributes
I'm confident that LightGBM does not shuffle the data. Row i in the Dataset you create refers to the same observation as row i in the provided raw data.
something is going wrong during fitting
I don't totally understand what you're trying to do here... this is a very large amount of provided code, and it's not clear to me which parts I should be looking at or what "something is going wrong during fitting" means specifically. If you could provide a much smaller example of what you're trying to accomplish, it'd help.
But even without that, I think I can still provide some useful resources.
First, see this example of using a custom objective function in the Python package: https://github.com/microsoft/LightGBM/blob/fe69fa9f43f6fc051730e65008184658c991106e/examples/python-guide/advanced_example.py#L139-L147. That's a classification example, but the interface is identical... you provide a function which, at each iteration, is given two things:
preds= predictions (in terms of the objective function, without transformation) for the ensemble so fartrain_data= thelightgbm.Datasetobject you're training on
and which is responsible for computing gradients and hessians.
It sounds to me like you want to be able to access other data inside such a custom objective function which is not features in the Dataset. I can think of a few options for doing that:
# option 1: make objective a callable class, store that other data on the class
class CustomObjective:
def __init__(self, time: np.ndarray):
self.time = time
def __call__(self, preds: np.ndarray, train_data: lightgbm.Dataset) -> Tuple[np.ndarray, np.ndarray]:
# your code that calculates the gradient and hessian
grad, hess = your_custom_code(preds, train_data, self.time)
return grad, hess
# option 2: store arbitrary data on the `Dataset` object itself
dtrain = lgb.Dataset(data=X, label=y, ...)
dtrain.time = time
def custom_objective(preds: np.ndarray, train_data: lightgbm.Dataset) -> Tuple[np.ndarray, np.ndarray]:
# your code that calculates the gradient and hessian
grad, hess = your_custom_code(preds, train_data, train_data.time)
return grad, hess
lgb.train(train_set=dtrain, params={"objective": custom_objective}, ...)See the note on writing custom objective functions at https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.train.html#lightgbm.train.
meh2135
Summary
Forward columns of your dataset directly to a custom objective function.
Motivation
This is useful for semi-parametric models like poisson process regression where y | x, t ~ Poisson(mu(x) * t) and we wish to learn mu(), or a variance regression where mean is known, ex: y | x, m ~ Logistic(m, exp(s(x)) and we wish to learn s().
It's possible that lightgbm would pick up on that model if you were to just feed it those parameters as features with no additional context, but consider the case where you're using the poisson process regression for causal inference, and the the time intervals are correlated with treatment status. Including them naively could lead to unpredictable biases.
Description
My optimal intferface would look something like this
I recognize that this is probably not widely useful enough to add as a feature to main, and might need some custom work for each use case, but I'm really struggling to make it work for my case. I've tried modifying the Dataset, the objective function wrapper, eval function wrapper, and inheriting from LGBMRegressor modifying the fit method to look for
locandlog_scaleparameters, as below, but something seems to be going wrong during fitting. My best guess is that there is some sorting by covariates happening at each boosting stage that doesn't apply to my hackily tacked on dataset attributes, so they end up misaligned with predictions and labels. I'm hoping that doesn't mean it requires digging into the c++...I would REALLY appreciate anyones advice on how to do this.