Closed vdrao closed 3 years ago
Hi @vdrao, the method you mention actually calculates the negative log likelihood (when using the LogScore
scoring rule). The correct sign is enforced in the definition of the score method for each distribution, e.g. for the normal.
Hello,
Thank you for NGBoost! I think there may be a bug in the sklearn score API in the main NGBoost class (lines 317-318):
Sklearn hyperparameter tuning functions like
RandomizedSearchCV
expect that better models are assigned higher (more positive) scores. However, as far as I can see, thetotal_score
method above actually computes the total log loss, which means that smaller values correspond to better models. Should the score instead be the negative log loss?