Custom metric object function

[karta282950]

Hi and thanks this awesome boosting. I want to ask how to custom metric object function in ngboost, like xgboost? (Ref. https://xgboost.readthedocs.io/en/latest/tutorials/custom_metric_obj.html)

[alejandroschuler]

Thanks @karta282950!

Using a custom metric (either for training or validation) is pretty easy but you need to know a little bit about the internals of ngboost to make it work how you want it to. The metric is controlled by an argument to the fit() method called train_loss_monitor or val_loss_monitor. The value of that argument needs to be a callable that itself accepts two arguments: a predicted distribution object D and a vector of targets Y. The exact nature of the distribution object depends on what Dist you constructed your ngboost model with. For example, let's say you're using Dist=Normal (the default) and you want to calculate mean absolute error (MAE) of the predicted mean during training. This is how I'd do it:

def mae_metric(D, Y):
    return np.mean(np.abs(D.loc - Y))

ngb = NGBRegressor(...)
ngb.fit(... , train_loss_monitor=mae_metric)

Notice I've referenced D.loc in the definition of mae_metric. This refers to the normal distribution's mean (or "location") parameter. What attributes/parameters the distribution has depends on the distribution you're using, and therefore so will the metric, in general. If you're just interested in the mean values of the distribution you can use D.predict(), which for regression distributions returns whatever parameter or function of parameters corresponds to the mean.

I'll close the issue for now but feel free to comment if you have further questions.

[karta282950]

@alejandroschuler Thank you for your help, i will try it.