Custom Brier and stopping criterion

[fipelle]

Hi, I am trying to implement a custom objective function equivalent to a custom Brier score for the classification. I suppose that I can implement the objective function following this approach. I just wanted to confirm though that objective(y_true, y_pred) -> grad, hess will look into the y_true and y_pred resulting from each potential split. Also, how can I pass some custom scalar (common across all potential splits, but different across trees) to the objective?

[jrvmalik]

You can wrap the objective in some class

import numpy as np

class Brier:

    def __init__(self):
        self.iteration: int = 0

    def objective(self, predictions: np.ndarray, targets_: np.ndarray) -> tuple[np.ndarray, np.ndarray]:
        self.iteration += 1
        return predictions - targets_, np.ones_like(predictions)

brier = Brier()
targets = np.zeros(30)
for idx in range(10):
    brier.objective(np.random.randn(30), targets)
    print(brier.iteration)

Or use this kind of sketchy functional approach

import numpy as np

def _objective(predictions, targets_):
    return predictions - targets_, np.ones_like(predictions)

_objective.iteration = 0

def objective(predictions, targets_):
    _objective.iteration += 1
    return _objective(predictions, targets_)

targets = np.zeros(30)
for idx in range(10):
    objective(np.random.randn(30), targets)
    print(_objective.iteration)