Hi, I hope to search for the model type as well (e.g., logistic regression or SVM), but I'm not sure it's correct to represent the model type as another dimension in the search space.
For example, suppose we have two types of models to choose type=[0, 1], and there is one hyperparemter associated with each of the two models alpha=[0,...,10]. The hyperparameters of those two different types of models have completely different meanings. For example, it's possible that with model type 0, it's better to use an alpha in range [2,3], while with model type 1, it's better to use an alpha in range [7,8]. If we treat the search space as a two dimensional space of [type, alpha], I suspect it's not going to work, because the target function is not continuous in this space.
Hi, I hope to search for the model type as well (e.g., logistic regression or SVM), but I'm not sure it's correct to represent the model type as another dimension in the search space.
For example, suppose we have two types of models to choose type=[0, 1], and there is one hyperparemter associated with each of the two models alpha=[0,...,10]. The hyperparameters of those two different types of models have completely different meanings. For example, it's possible that with model type 0, it's better to use an alpha in range [2,3], while with model type 1, it's better to use an alpha in range [7,8]. If we treat the search space as a two dimensional space of [type, alpha], I suspect it's not going to work, because the target function is not continuous in this space.
Do you know how to do that correctly?
Thank you!