Sometimes cant train the model and sometimes train the model although everything is the same LightGBM LambdaRank

[batutlsk]

I am working with LightGBM LambdaRank task and while i am training i realize something weird.

Please look below:

batuhan:Downloads batuhan.talsik$ lightgbm config=train.conf [LightGBM] [Info] Finished loading parameters [LightGBM] [Info] Loading query boundaries... [LightGBM] [Info] Total groups: 319, total data: 147860 [LightGBM] [Info] Loading query boundaries... [LightGBM] [Info] Total groups: 319, total data: 147860 [LightGBM] [Info] Finished loading data in 1.566017 seconds [LightGBM] [Info] Total Bins 3869 [LightGBM] [Info] Number of data: 147860, number of used features: 16 [LightGBM] [Info] Finished initializing training [LightGBM] [Info] Started training... [LightGBM] [Warning] No further splits with positive gain, best gain: -inf [LightGBM] [Warning] Stopped training because there are no more leaves that meet the split requirements [LightGBM] [Info] 0.545459 seconds elapsed, finished iteration 1 [LightGBM] [Info] Finished training batuhan:Downloads batuhan.talsik$ lightgbm config=train.conf [LightGBM] [Info] Finished loading parameters [LightGBM] [Info] Loading query boundaries... [LightGBM] [Info] Total groups: 319, total data: 147860 [LightGBM] [Info] Loading query boundaries... [LightGBM] [Info] Total groups: 319, total data: 147860 [LightGBM] [Info] Finished loading data in 1.502288 seconds [LightGBM] [Info] Total Bins 3869 [LightGBM] [Info] Number of data: 147860, number of used features: 16 [LightGBM] [Info] Finished initializing training [LightGBM] [Info] Started training... [LightGBM] [Warning] No further splits with positive gain, best gain: -inf [LightGBM] [Warning] Stopped training because there are no more leaves that meet the split requirements [LightGBM] [Info] 0.547467 seconds elapsed, finished iteration 1 [LightGBM] [Info] Finished training batuhan:Downloads batuhan.talsik$ lightgbm config=train.conf [LightGBM] [Info] Finished loading parameters [LightGBM] [Info] Loading query boundaries... [LightGBM] [Info] Total groups: 319, total data: 147860 [LightGBM] [Info] Loading query boundaries... [LightGBM] [Info] Total groups: 319, total data: 147860 [LightGBM] [Info] Finished loading data in 1.516244 seconds [LightGBM] [Info] Total Bins 3869 [LightGBM] [Info] Number of data: 147860, number of used features: 16 [LightGBM] [Info] Finished initializing training [LightGBM] [Info] Started training... [LightGBM] [Info] Iteration:1, training ndcg@10 : 0.66171 [LightGBM] [Info] Iteration:1, training map@10 : 0.613304 [LightGBM] [Info] Iteration:1, valid_1 ndcg@10 : 0.657377 [LightGBM] [Info] Iteration:1, valid_1 map@10 : 0.617258 [LightGBM] [Info] 0.586912 seconds elapsed, finished iteration 1 [LightGBM] [Info] Iteration:2, training ndcg@10 : 0.712599 [LightGBM] [Info] Iteration:2, training map@10 : 0.667139 [LightGBM] [Info] Iteration:2, valid_1 ndcg@10 : 0.686665 [LightGBM] [Info] Iteration:2, valid_1 map@10 : 0.651572 [LightGBM] [Info] 1.187747 seconds elapsed, finished iteration 2 [LightGBM] [Info] Iteration:3, training ndcg@10 : 0.724263 [LightGBM] [Info] Iteration:3, training map@10 : 0.683256 [LightGBM] [Info] Iteration:3, valid_1 ndcg@10 : 0.714376 [LightGBM] [Info] Iteration:3, valid_1 map@10 : 0.687333 [LightGBM] [Info] 1.790020 seconds elapsed, finished iteration 3 [LightGBM] [Info] Iteration:4, training ndcg@10 : 0.729973 [LightGBM] [Info] Iteration:4, training map@10 : 0.690267 [LightGBM] [Info] Iteration:4, valid_1 ndcg@10 : 0.711804 [LightGBM] [Info] Iteration:4, valid_1 map@10 : 0.686864

and so on..

First try, cant train the model, second try, cant train the model and then third try i can train the model. Is there anyone who knows the reason of that ? Same data, same config, same command.

Train config :

{ 'task':'train', 'objective':'lambdarank', 'boosting_type' : 'dart', 'max_depth':5, 'num_leaves': 14, 'metric': ['ndcg','map'], 'eval_at':10, 'max_bin':255, 'metric_freq' : 1, 'num_trees':500, 'learning_rate':0.05, 'bagging_fraction':0.3, 'bagging_freq':1, 'min_data_in_leaf':20, 'num_threads':8 }

Can anyone help me ?

[guolinke]

firstly, did you try in on the latest master branch? and did you fix the seed, and num_threads ? I will suggest you to use gbdt instead of dart, and try non-bagging to test.

[batutlsk]

i tried all of this but problem is still exists.

[batutlsk]

Someone else has a different idea ?

[guolinke]

could you provide the reproduce example? I guess it is more like the env/usage problem, if above solutions didn't help.

[batutlsk]

Tree=0 num_leaves=1 num_cat=0 split_feature= split_gain= threshold= decision_type= left_child= right_child= leaf_value=0 leaf_weight=0 leaf_count=0 internal_value= internal_weight= internal_count= shrinkage=1

end of trees

I realized that when [LightGBM] [Warning] Stopped training because there are no more leaves that meet the split requirements occurs, model is like above. num_leaves=1 and shrinkage=1 . Is there a something like 'boost_from_average=false' for lambdarank ?

[guolinke]

Yeah, it is expected, as nothing is learned from your data. shrinkage is always 1 for the dummy trees.

[StrikerRUS]

ping @batutlsk for

could you provide the reproduce example?

[batutlsk]

Now, works with no problems but i didn't change anything

[guolinke]

feel free to reopen if you met it again.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.