search

brandontrabucco / design-bench

Benchmarks for Model-Based Optimization
MIT License
80 stars 19 forks source link

Summary of the characteristics of the datasets? #6

Open sgbaird opened 2 years ago

sgbaird commented 2 years ago

For example, dimensionality, number of numerical vs. categorical features, and if possible, some notion of complexity for each of the tasks (an example would be a comparison against random search performance)

brandontrabucco commented 2 years ago

Hi sgbaird,

Thanks for your interest in our benchmark! You can get a detailed review of the datasets contained in the benchmark that we are using in our research in our whitepaper here: https://arxiv.org/abs/2202.08450.

-Brandon

sgbaird commented 2 years ago

Thanks, found the information I was looking for. I noticed that the focus is largely on one-shot (offline) design. Do you think these are suitable benchmarks for adaptive design (online) versions as well?