I would like to report a potential issue. I have Python 3.6.5 with scikit-learn 0.20.2 and ngboost 0.3.10.
I used the following code:
_____
import ngboost
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.datasets import load_boston
from sklearn.model_selection import train_test_split
ryan-wolbeck
commented
3 years ago
Hi all,
I would like to report a potential issue. I have Python 3.6.5 with scikit-learn 0.20.2 and ngboost 0.3.10.
I used the following code:
_____
import ngboost from sklearn.ensemble import GradientBoostingRegressor from sklearn.datasets import load_boston from sklearn.model_selection import train_test_split
x, y= load_boston(return_X_y= True)
y= y.astype(np.float32)
x= ((x - np.mean(x, axis= 0)) / np.std(x, axis= 0)).astype(np.float32)
x_train, x_validation, y_train, y_validation= train_test_split(x, y, test_size= 0.4, random_state= 1969)
learner= GradientBoostingRegressor(loss= 'lad', learning_rate= 0.25, n_estimators= 1, criterion= 'mse', max_depth= 6, min_impurity_decrease= 0, random_state= 1969)
ngb= ngboost.NGBRegressor(Dist= ngboost.distns.Normal, Score= ngboost.scores.CRPScore, Base= learner, natural_gradient= True, n_estimators= 300, learning_rate= 0.01, verbose= False, random_state= 1969)
ngb.fit(x_train, y_train, X_val= x_validation, Y_val= y_validation)
y_preds_1= ngb_1.predict(x_validation)
_____
The above code gives this runtime warning message:
c:\temp\python\python3.6.5\lib\site-packages\ngboost\distns\normal.py:71: RuntimeWarning: overflow encountered in square self.var = self.scale ** 2
To avoid this message, I have to set "learning_rate= 0.1" (or any value lower than the original 00.25) in GradientBoostingRegressor call command
Ivan