automl / neps

Neural Pipeline Search (NePS): Helps deep learning experts find the best neural pipeline.
https://automl.github.io/neps/
Apache License 2.0
53 stars 13 forks source link

Accidental Prior Usage Activation Due to ConstantParameter in SearchSpace #44

Closed danrgll closed 9 months ago

danrgll commented 10 months ago

Accidental Prior Usage Activation Due to ConstantParameter in SearchSpace if another Parameter has is_fidelity=True

Why does this error occur:

src/neps/search_spaces/hyperparameters/constant.py

class ConstantParameter(NumericalParameter): def init(self, value: Union[float, int, str], is_fidelity: bool = False): super().init() self.value = value self.is_fidelity = is_fidelity self.default = value # causing issue

src/neps/search_spaces/search_space.py

SearchSpace() def init(): .....

Check if defaults exists to construct prior from

          if hasattr(hyperparameter, "default") and hyperparameter.default is not None:
                  self.has_prior = True
        elif hasattr(hyperparameter, "has_prior") and hyperparameter.has_prior:
                  self.has_prior = True

Proposed Fix: if hasattr(hyperparameter, "default") and hyperparameter.default is not None: if not isinstance(hyperparameter, ConstantParameter): self.has_prior = True elif hasattr(hyperparameter, "has_prior") and hyperparameter.has_prior: self.has_prior = True

But: Still issue(warning) with searcher Hyperband

/src/neps/optimizers/multi_fidelity/successive_halving.py:240: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation. self.observed_configs = pd.concat(