elephaint
commented
2 years ago Hi,
It looks like a genuine bug that I'd have to reproduce. Most likely, hyperparameter combination may cause this. I'd suggest min_data_in_leaf > 2 always, but that does not necessarily have to relate to this issue. If you can share the script and data that I'd be happy to delve further into it next week. Sorry to not be able to help you further at this moment!
In the sklearn wrapper, you should use reg_lambda instead of lambda (see here: https://github.com/elephaint/pgbm/blob/65b6d0331a70520ea8b545cbd6196133c4a7c11b/src/pgbm/pgbm.py#L1179) Sorry for the incovenience, but otherwise there are issues with Python's lambda function.
Random_seed is random_state in the sklearn wrapper to adhere to sklearn naming conventions (see here: https://github.com/elephaint/pgbm/blob/65b6d0331a70520ea8b545cbd6196133c4a7c11b/src/pgbm/pgbm.py#L1182)
Best,
Olivier
From: flippercy @.> Sent: Wednesday, September 22, 2021 4:44:17 PM To: elephaint/pgbm @.> Cc: Olivier Sprangers @.>; Mention @.> Subject: [elephaint/pgbm] An error with PGBM (#7)
Hi @elephainthttps://github.com/elephaint:
I got the following error when using the sklearn wrapper, PGBMRegressor:
~/.local/lib/python3.7/site-packages/pgbm/pgbm.py in _predict_tree(self, X, mu, variance, estimator) 401 # Choose next node (based on breadth-first) 402 condition = (nodes_predict >= node) * (predictions == 0) --> 403 node = nodes_predict[condition].min() 404 # Select current node information 405 split_node = nodes_predict == node
RuntimeError: min(): Expected reduction dim to be specified for input.numel() == 0. Specify the reduction dim with the 'dim' argument.
It is not due to the data because I used the same data to build a PGBM model successfully before. Is it due to some hyperparameters? I got this error when trying to do HPO for PGBM using FLAML (https://github.com/microsoft/FLAML) and the search space I used is:
'max_bin': {'domain': tune.loguniform(lower=32, upper=32767), 'init_value': 256, 'low_cost_init_value': 256}, 'max_leaves': {'domain': tune.uniform(lower=16, upper=128), 'init_value': 64}, 'n_estimators': {'domain': tune.uniform(lower = 50, upper = 500), 'init_value': 200, 'low_cost_init_value': 200}, 'min_data_in_leaf': {'domain': tune.uniform(lower = 1, upper = 1000), 'init_value': 100, 'low_cost_init_value': 100}, 'bagging_fraction': {'domain': tune.uniform(lower = 0.6, upper = 1), 'init_value': 0.7, 'low_cost_init_value': 0.7}, 'feature_fraction': {'domain': tune.uniform(lower = 0.5, upper = 1), 'init_value': 0.9, 'low_cost_init_value': 0.9}, 'learning_rate': {'domain': tune.loguniform(lower = 0.001, upper = 1), 'init_value': 0.1, 'low_cost_init_value': 0.1}, 'min_split_gain': {'domain': tune.loguniform(lower = 0.000000000001, upper = 0.001), 'init_value': 0.00001, 'low_cost_init_value': 0.00001},
By the way, it seems that 'lambda' and 'seed' are missing in the sklearn wrapper.
Thank you.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/elephaint/pgbm/issues/7, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKYHS4FV4QA3NLXUX2AA6O3UDHTUDANCNFSM5ERP5AJA. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
flippercy
Hi @elephaint:
I got the following error when using the sklearn wrapper, PGBMRegressor:
~/.local/lib/python3.7/site-packages/pgbm/pgbm.py in _predict_tree(self, X, mu, variance, estimator) 401 # Choose next node (based on breadth-first) 402 condition = (nodes_predict >= node) * (predictions == 0) --> 403 node = nodes_predict[condition].min() 404 # Select current node information 405 split_node = nodes_predict == node
RuntimeError: min(): Expected reduction dim to be specified for input.numel() == 0. Specify the reduction dim with the 'dim' argument.
Any insight? It is not due to the data because I used the same data to build a PGBM model successfully before. Is it due to some hyperparameters? I got this error when trying to do HPO for PGBM using FLAML (https://github.com/microsoft/FLAML) and the search space I used is:
'max_bin': {'domain': tune.loguniform(lower=32, upper=32767), 'init_value': 256, 'low_cost_init_value': 256}, 'max_leaves': {'domain': tune.uniform(lower=16, upper=128), 'init_value': 64}, 'n_estimators': {'domain': tune.uniform(lower = 50, upper = 500), 'init_value': 200, 'low_cost_init_value': 200}, 'min_data_in_leaf': {'domain': tune.uniform(lower = 1, upper = 1000), 'init_value': 100, 'low_cost_init_value': 100}, 'bagging_fraction': {'domain': tune.uniform(lower = 0.6, upper = 1), 'init_value': 0.7, 'low_cost_init_value': 0.7}, 'feature_fraction': {'domain': tune.uniform(lower = 0.5, upper = 1), 'init_value': 0.9, 'low_cost_init_value': 0.9}, 'learning_rate': {'domain': tune.loguniform(lower = 0.001, upper = 1), 'init_value': 0.1, 'low_cost_init_value': 0.1}, 'min_split_gain': {'domain': tune.loguniform(lower = 0.000000000001, upper = 0.001), 'init_value': 0.00001, 'low_cost_init_value': 0.00001},
Thank you.