Closed karta282950 closed 3 years ago
Thanks @karta282950!
Using a custom metric (either for training or validation) is pretty easy but you need to know a little bit about the internals of ngboost to make it work how you want it to. The metric is controlled by an argument to the fit()
method called train_loss_monitor
or val_loss_monitor
. The value of that argument needs to be a callable that itself accepts two arguments: a predicted distribution object D
and a vector of targets Y
. The exact nature of the distribution object depends on what Dist
you constructed your ngboost model with. For example, let's say you're using Dist=Normal
(the default) and you want to calculate mean absolute error (MAE) of the predicted mean during training. This is how I'd do it:
def mae_metric(D, Y):
return np.mean(np.abs(D.loc - Y))
ngb = NGBRegressor(...)
ngb.fit(... , train_loss_monitor=mae_metric)
Notice I've referenced D.loc
in the definition of mae_metric
. This refers to the normal distribution's mean (or "location") parameter. What attributes/parameters the distribution has depends on the distribution you're using, and therefore so will the metric, in general. If you're just interested in the mean values of the distribution you can use D.predict()
, which for regression distributions returns whatever parameter or function of parameters corresponds to the mean.
I'll close the issue for now but feel free to comment if you have further questions.
@alejandroschuler Thank you for your help, i will try it.
Hi and thanks this awesome boosting. I want to ask how to custom metric object function in ngboost, like xgboost? (Ref. https://xgboost.readthedocs.io/en/latest/tutorials/custom_metric_obj.html)