stanfordmlgroup / ngboost

Natural Gradient Boosting for Probabilistic Prediction
Apache License 2.0
1.65k stars 217 forks source link

overflow warnings in NormalCRPScore #87

Closed alejandroschuler closed 4 years ago

alejandroschuler commented 4 years ago
X, Y = load_boston(True)
X_reg_train, X_reg_test, Y_reg_train, Y_reg_test = train_test_split(X, Y, test_size=0.2)
ngb = NGBRegressor(Dist=Normal, Score=CRPScore).fit(X_reg_train, Y_reg_train)

gives a bunch of warnings like RuntimeWarning: overflow encountered in exp self.scale = np.exp(params[1]), but at the end of the day the metrics seem ok:

Y_preds = ngb.predict(X_reg_test)
Y_dists = ngb.pred_dist(X_reg_test)

# test Mean Squared Error
test_MSE = mean_squared_error(Y_preds, Y_reg_test)
print('Test MSE', test_MSE)

# test Negative Log Likelihood
test_NLL = -Y_dists.logpdf(Y_reg_test).mean()
print('Test NLL', test_NLL)

>>> Test MSE 15.96052771978117
>>> Test NLL 2.520399808781495
ryan-wolbeck commented 4 years ago

@alejandroschuler I think this usually occurs when the value doesn't fit in the default numpy float64 which makes sense with an exponential. I don't think this will cause any issue other than the annoying warnings