Questions re: documentation for optimise() method

[jimthompson5802]

The optimise documentation page contains this passage:

IMPORTANT : Try to avoid dependent parameters and to set one feature 
selection strategy and one estimator strategy at a time.

• I'm not sure what is meant by the phrase "Try to avoid dependent parameters...". An example of will be helpful.

• Is this a correct interpretation of "...set one feature selection strategy and one estimator as at a time." the following:

  
  space1 = {
      'ne__numerical_strategy':{"search":"choice",
                               "space":['median']},
      'ce__strategy':{"search":"choice",
                      "space":["label_encoding","random_projection", "entity_embedding"]}, 
     'fs_strategy' : {'search': 'choice', 'space':['l1']},
      'fs__threshold':{"search":"uniform",  "space":[0.01,0.3]},    
      'est__strategy': {'search': 'choice',  'space': ['LightGBM']},
      'est__learning_rate': {'search': "uniform", 'space': [0.001,0.05]},
      'est__max_depth':{"search":"choice",  "space":[3,5, 7, 9, 11, 13, 15]},
      'est__n_estimators': {'search':'choice', 'space':[50, 100, 150, 200, 400, 800, 1200]}
      }

best1 = opt.optimise(space1,df)

space2 = { 'nenumerical_strategy':{"search":"choice", "space":['median']}, 'cestrategy':{"search":"choice", "space":["label_encoding","random_projection", "entity_embedding"]}, 'fs_strategy' : {'search': 'choice', 'space':['rf_feature_importance']}, 'fsthreshold':{"search":"uniform", "space":[0.01,0.3]},
'eststrategy': {'search': 'choice', 'space': ['RandomForest']}, 'estlearning_rate': {'search': "uniform", 'space': [0.001,0.05]}, 'estmax_depth':{"search":"choice", "space":[3,5, 7, 9, 11, 13, 15]}, 'est__n_estimators': {'search':'choice', 'space':[50, 100, 150, 200, 400, 800, 1200]} }

best2 = opt.optimise(space2,df)

[AxeldeRomblay]

Hello @jimthompson5802, Sorry for the late answer, I'm quite busy these days... Yes you're right, you should avoid optimizing two (or more) different models (eg: RandomForest & LightGBM) that share common parameters (eg : max_depth). The optimizer could be lost when trying different values for the different models... (max_depth = 10 for a RandomForest is not equivalent for a LightGBM...). But once again, it will work but will be less accurate I guess...