LightGBM Warning when strategy is Linear

[Nithanaroy]

I'm getting error saying the parameter is unknown for a LightGBM classifier when I chose est__strategy as Linear. However the log shows that LogisticRegression was picked for HP search. I'm confused from the log. What am I missing? Log

##################################################### testing hyper-parameters... #####################################################

>>> NA ENCODER :{'numerical_strategy': 'mean', 'categorical_strategy': 'most_frequent'}

>>> CA ENCODER :{'strategy': 'entity_embedding'}

>>> FEATURE SELECTOR :{'strategy': 'l1', 'threshold': 0.25183075829772344}

>>> ESTIMATOR :{'strategy': 'Linear', 'C': 1.0, 'class_weight': None, 'dual': False, 'fit_intercept': True, 'intercept_scaling': 1, 'max_iter': 100, 'multi_class': 'ovr', 'n_jobs': -1, 'penalty': 'l2', 'random_state': 0, 'solver': 'liblinear', 'tol': 0.0001, 'verbose': 0, 'warm_start': False}

/opt/conda/lib/python3.6/site-packages/mlbox/model/classification/classifier.py:92: UserWarning: Invalid parameter for classifier LightGBM. Parameter IGNORED. Check the list of available parameters with `classifier.get_params().keys()`
  + ". Parameter IGNORED. Check the list of "
/opt/conda/lib/python3.6/site-packages/mlbox/model/classification/classifier.py:92: UserWarning: Invalid parameter for classifier LightGBM. Parameter IGNORED. Check the list of available parameters with `classifier.get_params().keys()`
  + ". Parameter IGNORED. Check the list of "
/opt/conda/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:1228: UserWarning: 'n_jobs' > 1 does not have any effect when 'solver' is set to 'liblinear'. Got 'n_jobs' = -1.
  " = {}.".format(self.n_jobs))
/opt/conda/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:1228: UserWarning: 'n_jobs' > 1 does not have any effect when 'solver' is set to 'liblinear'. Got 'n_jobs' = -1.
  " = {}.".format(self.n_jobs))
/opt/conda/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:1228: UserWarning: 'n_jobs' > 1 does not have any effect when 'solver' is set to 'liblinear'. Got 'n_jobs' = -1.
  " = {}.".format(self.n_jobs))

Code

space = {    
    'ne__numerical_strategy': {
        "search": "choice", 
        "space": ['mean']
    },
    'ne__categorical_strategy': {
        "search": "choice", 
        "space": ['most_frequent']
    },

    'ce__strategy': {
        "search": "choice",
        "space": ["label_encoding", "random_projection", "entity_embedding"]
    }, 

    # https://mlbox.readthedocs.io/en/latest/features.html#feature-selection
    'fs__strategy': {
        "search": "choice",
        "space": ['l1']
    },
    'fs__threshold':{
        "search":"uniform",
        "space":[0.01, 0.3]
    },  

    # https://mlbox.readthedocs.io/en/latest/features.html#id1
    'est__strategy': {
        "search": "choice",
        "space": ["Linear"]
    },

    # Params: https://github.com/AxeldeRomblay/MLBox/blob/master/mlbox/model/classification/classifier.py#L103
    # https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html
    'est__penalty': {
        "search": "choice",
        "space": ["l2"]
    },
    'est__max_iter': {
        "search": "choice",
        "space": [100, 200, 300, 400, 500, 800]
    }
}

max_trials_from_space = 60 # max number of times samples from above space are taken
best_lin = opt.optimise(space, data, max_trials_from_space)

[AxeldeRomblay]

You wrote : est__stratagy instead of est__strategy so the strategy will remain a LightGBM and MLBox is looking for a parameter "stratagy" for LightGBM which does not exist...

[Nithanaroy]

Axel, that was a typo in the problem description. In the config below it is correct which describe thr search space it is correct.