General list of learner parameter peculiarities

[mb706]

This is a list of things that I noticed while working with some learners. I'm no expert on any of the learners, but the documentation or my intuition told me these might be bugs. Maybe some of these things were already fixed in the meantime. If someone wants to double-check and fix some of these...

All these are classif.xxx, but may also apply to the regr. etc. equivalent

• [ ] blackboost
• blackboost: Binomial.link depends on family==Binomial
• [ ] glmboost
• glmboost: same
• [ ] bst
• bst: 'nu' says -Inf to Inf, but doc says 0..1
• bst: bst_control has a few more parameters that are not in the param set
• bst: rpart control parameters should require Learner == tree, 'df' require Learner == sm
• [ ] xgboost
• xgboost: eta, gamma, max_depth, min_childweight, subsample, colsample*, num_parallel_tree: depend on booster %in% c('gbtree', 'dart') (?)
• xgboost: (note: lambda, lambda_bias, alpha apparently do not depened on booster value)
• xgboost: why is 'base_score' not tunable?
• xgboost: why is 'nrounds' default 1?
• xgboost: 'objective' should default to 'null'? b/c using default='binary:logistic' is different behaviour from giving objective=NULL
• [ ] bartMachine
• bartMachine: num_iterations_after_burn_in must be > 1
• bartMachine: alpha \in (0, 1)
• classif.bartMachine: param q 'not used for classification'
• bartMachine: mh_prob_steps > 0
• bartmachine: run_in_sample, serialize, mem_cache_for_speed, (num_rand_samps_in_library?) not tunable
• [ ] randomForest
• randomForest: should not be 'tunable'
• randomForest: oob.prox defaults to 'FALSE'
• [ ] extraTrees
• extraTrees: limit for nodesize, numRandomCuts, subsetSizes is PROBABLY not -Inf, but rather 1
• extraTrees: numThreads, tasks, probOfTaskCuts not tunable
• [ ] randomForestSRC
• randomForestSRC: nodedepth: -1 gives the default behaviour, but is not the default as given in the function head of rfsrc. also, is the range really from -Inf?
• [ ] ranger
• ranger: 'respect.unordered.factors' has more values than TRUE/FALSE
• [ ] rFerns
• rFerns: 'depth' is 1..16, its in the manual guys
• [ ] rotationForest
• rotationForest: rpart arguments
• [ ] nodeHarvest
• nodeHarvest: 'addto" maybe not tunable?

EDIT 2017-03-21: additional things I noticed until now

• [ ] rrlda
• rrlda: hp parameter probably ranges from 0 to 1
• [ ] mda
• mda: 'subclasses' default is wrong
• mda: kee.fitted default is TRUE only if n * dimension < 5000
• [ ] quaDA
• quaDA: validation "learntest" doesn't work (will always crash), since there is no way to specify train / test split.
• [ ] kknn
• kknn: kernel param is missing 'rank' option
• kknn: 'distance' parameter lower bound is exclusive: distance == 0 gives an error.
• [ ] dcSVM
• dcSVM: the defaults for 'm' and 'max.levels' is clearly 800, and 1. (Even if the SwarmSVM::dcSVM doesn't see it like this, the trainLearner function does it.)
• dcSVM: to add: this system is clearly broken as it is. instead, the params should be set to 1 and 800 during makeLearner().
• dcSVM: cluster.method "mlKmeans" is also an option
• dcSVM: some of the params are probably (?) not 'tunable' e.g. seed
• [ ] clusterSVM [ ]
• clusterSVM: cluster.method mlKmeans missing
• clusterSVM: some of the params are probably (?) not 'tunable' e.g. seed
• clusterSVM: 'type' parameter seems to be the type parameter of LiblineaR, which seems to accept type 1 to 7 for classification, instead of 1 to 5. (?)
• [ ] gaterSVM
• gaterSVM: default of 'max.iter' is not really 1. it is set to 1 by makeLearner, but that is different.
• gaterSVM: learningrate lower bound is 1e-8
• [ ] ksvm
• ksvm: epsilon parameter seems to matter only for regression, can be removed
• ksvm: 'nu' is bounded between 0 and 1
• ksvm: 'order' may not be < 0.
• [ ] dbnDNN
• dbnDNN: I think 'cd' should be >= 1. The learner doesn't crash with cd < 1, but I think that's because it uses a '1:cd' loop; basically a glitch.
• dbnDNN: batchsize == 1 seems to not work, so maybe 2 is the lower limit? (though this may be an internal bug where they forgot to 'drop=FALSE')
• [ ] nnTrain
• nnTrain: batchsize == 1 seems to not work, so maybe 2 is the lower limit? (though this may be an internal bug where they forgot to 'drop=FALSE')
• [ ] neuralnet
• neuralnet: learningrate is 'only used by traditional backpropagation' according to help(). so maybe add requirement algorithm == "backprop"
• neuralnet: learningrate.factor, learningrate.limit 'used only for RPROP and GRPROP', so add requirement 'algorithm != "backprop"'
• neuralnet: learningrate.factor, learningrate.limit appear to always be of length 2, and should probably be of type NumericVector, so add that to paramset info
• [ ] nnet
• nnet: 'size' parameter minimum is 1
• nnet: weight decay minimum is maybe 0? The learner does run with weight decay < 0, but does it make sense?
• [ ] mlp
• mlp: initFunc, learnFunc, etc (all the ***Func params). could have constraints set to the various offered init / learn functions offered by the package
• [ ] JRip
• JRip: not sure: F requires P == FALSE ?
• [ ] earth
• earth: param linpreds has arbitrary length
• earth: ncross requires nfold > 1 && pmethod == 'cv'
• earth: ncross must be between 1 and 1000, inclusive
• earth: stratify requires nfold > 1L && pmethod == "cv" (the second part is missing)
• earth: adjust.endspan must be between 0 and 10
• earth: Exhaustive.tol requires pmethod == "exhaustive"
• [ ] gamboost
• gamboost: 'offset' is supposed to be an arbitrary length vector?
• gamboost: dfbase minimum is 2?
• gamboost: dfbase requires baselearner == "bbs"
• [ ] RFF
• RFF: coefReg must be between 0 and 1
• RRF: nPerm 'currently only implemented for regression' according to help

Other things:

• mention 'listLearners' in makeLearner help page (it is referred to indirectly by mentioning the 'learners' help page, but this help page is a three line page that only refers to listLearners and to online help.

[ja-thomas]

Thanks a lot for that list!

[berndbischl]

@MariaErdmann can you please iterate over at least some and check?

[MariaErdmann]

Yes, will do this together when fixing learners from #1370

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.