search

martinjankowiak / saasbo

SAASBO: a package for high-dimensional bayesian optimization
38 stars 5 forks source link

Are you aware of a Bayesian optimization model that uses recursive feature elimination (RFE)? #2

Open sgbaird opened 2 years ago

sgbaird commented 2 years ago

@dme65 @martinjankowiak,

I enjoyed going through your SAASBO paper, especially the thorough benchmarking and comparisons. Are you aware of a Bayesian optimization model that uses recursive feature elimination (RFE)? If so, is it any good?

The general idea being:

  1. Generate SOBOL points
  2. Loop
    1. Fit a GP to the full dataset
    2. add next suggested BO points
    3. recursively remove least important features

Sterling

martinjankowiak commented 2 years ago

@sgbaird not sure what you have in mind but it sounds sort of like

Cheng Li, Sunil Gupta, Santu Rana, Vu Nguyen, Svetha Venkatesh, and Alistair Shilton. High dimensional Bayesian optimization using dropout

from my perspective a good feature of something like saasbo is that no hard decisions are made about feature inclusion/exclusion. different posterior samples may, in effect, include different sets of features. the acquisition function can then use that uncertainty when suggesting the next query point

sgbaird commented 2 years ago

Thank you! This was pretty similar. I appreciate the comment about not leaving out features with SAASBO.