Query about 'initial_design' argument in SMAC usage

[SantoshParit]

I have been exploring the documentation for SMAC (Sequential Model-based Algorithm Configuration) and came across a particular argument named 'initial_design'. I have a question regarding its usage and implications in the context of optimizing hyperparameters.

In my reading of code and document, I noticed that 'initial_design' is utilized in some of the source code examples provided in the SMAC3 repository. The code snippet I found is as follows:

# We want to run the facade's default initial design, but we want to change the number
# of initial configs to 5.
initial_design = HyperparameterOptimizationFacade.get_initial_design(scenario, n_configs=5)

# Now we use SMAC to find the best hyperparameters
smac = HyperparameterOptimizationFacade(
    scenario,
    classifier.train,
    initial_design=initial_design,
    overwrite=True,  # If the run exists, we overwrite it; alternatively, we can continue from the last state
)

incumbent = smac.optimize()

My primary question is whether the 'initial_design' argument is intended to be used as a way to warm-start the optimization process. Specifically, can the 'initial_design' argument be utilized to provide hand-designed hyperparameter values that guide the initial stages of the optimization? In other words, does supplying custom hyperparameter values through 'initial_design' allow the Bayesian optimization process to start from a set of predefined configurations?

If the 'initial_design' argument doesn't serve this purpose, I'm interested in knowing if SMAC3 offers any feature that enables providing starting hyperparameter values. I'm looking for a functionality where I can supply a set of initial configurations, and based on these values, the Bayesian optimization process guides the selection of subsequent points for evaluation.

Your insights into this matter would greatly assist me in better understanding the capabilities of SMAC3 and its usage for hyperparameter optimization. I appreciate your time and assistance.

Thank you for your attention, and I look forward to hearing from you.

Best regards, Santosh

[dengdifan]

Hi, If you would like to customise the initial design, you can add them through: https://github.com/automl/SMAC3/blob/main/smac/initial_design/abstract_initial_design.py#L54 and set https://github.com/automl/SMAC3/blob/main/smac/initial_design/abstract_initial_design.py#L51 as 0. Then SMAC will only start with the initial design from additional_configs. Please let me know if that helps. Best regards, Difan

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.