[ERROR]:ciir.umass.edu.utilities.RankLibError

[Gu-Youngfeng]

The newly-submitted change in reconfig.py added more tunable parameters, such as -ranker, but this change can induce some error, the failing trace when running the reconfig.py is as follows,

STEP-5: build the L2R model to re-rank the prediction of test set ...

../temp_data/txt/trainset_txt/Apache_AllMeasurements
Exception in thread "main" ciir.umass.edu.utilities.RankLibError: Unknown command-line parameter: unspecified
        at ciir.umass.edu.utilities.RankLibError.create(RankLibError.java:26)
        at ciir.umass.edu.eval.Evaluator.main(Evaluator.java:409)

How did this happen? Are there any parameter set to a wrong value?

[Gu-Youngfeng]

Perhaps the key factor of this error is caused by the last change, here is the code snippet in reconfig.py, The variable param_mid_cmd provides the acceptable ranker list in the RankLib.jar. However, when running RankLib.jar based on one of the provided rankers, some string-type-parameter CANNOT by recognised, such as -reg no-regularization and -noeq unspecified.

param_ranker  = [0, 1, 2, 3, 4, 6, 7, 8]   
param_mid_cmd = [" -r 5 -i 25 -tolerance 0.001 -reg no-regularization", # 0-MART
         " -epoch 100 -layer 1 -node 10 -lr 0.00005", # 1-RankNet
         " -round 300 -tc 10", # 2-RankBoost
         " -metric2t ERR@10 -round 500 -noeq unspecified -tolerance 0.002 -max 5", # 3-AdaRank (list-wise)
         " -metric2t ERR@10 -r 5 -i 25 -tolerance 0.001 -reg no-regularization", # 4-Coordinate Ascent (list-wise)
         " -metric2t ERR@10 -r 5 -i 25 -tolerance 0.001 -reg no-regularization", # 6-LambdaMart (list-wise)
         "", # 7-ListNet (list-wise)
         " -bag 300 -srate 1.0 -frate 0.3 -rtype 0 -tree 1 -leaf 100 -shrinkage 0.1"] # 8-Random Forest

One of the simplest solution is to remove these string-type-parameters in param_mid_cmd, the reasonable patches of this issue is like,

param_ranker  = [0, 1, 2, 3, 4, 6, 7, 8]
param_mid_cmd = [" -r 5 -i 25 -tolerance 0.001", # 0-MART
                 " -epoch 100 -layer 1 -node 10 -lr 0.00005", # 1-RankNet
                 " -round 300 -tc 10", # 2-RankBoost
                 " -metric2t ERR@10 -round 500 -tolerance 0.002 -max 5", # 3-AdaRank (list-wise)
                 " -metric2t ERR@10 -r 5 -i 25 -tolerance 0.001 ", # 4-Coordinate Ascent (list-wise)
                 " -metric2t ERR@10 -r 5 -i 25 -tolerance 0.001 ", # 6-LambdaMart (list-wise)
                 "", # 7-ListNet (list-wise)
                 " -bag 300 -srate 1.0 -frate 0.3 -rtype 0 -tree 1 -leaf 100 -shrinkage 0.1"] # 8-Random Forest