Ordinal parameters with conditions fails to impute in the random forest model

[EdvardHolden]

Hi,

SMAC: 2.0.2 (installed via pip) Python: 3.11

Description

I am working on updating a tool from using SMACv2 (Java) to SMACv3, and support for conditions on ordinal parameters seems to have been removed. If the condition for the ordinal parameter is removed or it is made categorical, everything seems to work as expected.

I have not found any documentation indicating that conditions on ordinals are deprecated/removed in the new version.

Code to Reproduce

from ConfigSpace import Configuration, ConfigurationSpace, Categorical, EqualsCondition
from smac import HyperparameterOptimizationFacade, Scenario

def dummy_target(config: Configuration, seed: int = 0) -> float:
    return 1.0

def main() -> None:
    configspace = ConfigurationSpace()

    # Define parameters
    res_flag = Categorical("resolution_flag", ["true", "false"], ordered=False, default="true")
    res_comb = Categorical("resolution_comb", [1, 2, 4, 8, 16, 32], ordered=True, default=1)
    configspace.add_hyperparameters([res_flag, res_comb])
    # Define constraints
    res_comb_cond = EqualsCondition(res_comb, res_flag, "true")
    configspace.add_condition(res_comb_cond)
    print(configspace)

    # Get the scenario and run SMAC
    scenario = Scenario(configspace, deterministic=True, n_trials=3, n_workers=1, use_default_config=True)
    # Use SMAC to find the best configuration/hyperparameters
    smac = HyperparameterOptimizationFacade(scenario, target_function=dummy_target, overwrite=True)

    smac.optimize()

if __name__ == "__main__":
    main()

Expected Results

Expect the random forest model to train successfully and generate configurations, as in the Java version.

Actual Results

Result: ValueError. Imputing the ordinal value fails as there is no type check for OrdinalHyperparameter in the if statement, and the else block raises a ValueError.

[...]
[INFO][abstract_intensifier.py:515] Added config e8401b as new incumbent because there are no incumbents yet.
Traceback (most recent call last):
  File "/home/eholden/sandbox/test_smac/run_ordinal.py", line 30, in <module>
    main()
  File "/home/eholden/sandbox/test_smac/run_ordinal.py", line 26, in main
    smac.optimize()
  File "/home/eholden/.pyenv/versions/3.11.3/lib/python3.11/site-packages/smac/facade/abstract_facade.py", line 319, in optimize
    incumbents = self._optimizer.optimize(data_to_scatter=data_to_scatter)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eholden/.pyenv/versions/3.11.3/lib/python3.11/site-packages/smac/main/smbo.py", line 300, in optimize
    trial_info = self.ask()
                 ^^^^^^^^^^
  File "/home/eholden/.pyenv/versions/3.11.3/lib/python3.11/site-packages/smac/main/smbo.py", line 153, in ask
    trial_info = next(self._trial_generator)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eholden/.pyenv/versions/3.11.3/lib/python3.11/site-packages/smac/intensifier/intensifier.py", line 226, in __iter__
    config = next(self.config_generator)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eholden/.pyenv/versions/3.11.3/lib/python3.11/site-packages/smac/main/config_selector.py", line 190, in __iter__
    self._model.train(X, Y)
  File "/home/eholden/.pyenv/versions/3.11.3/lib/python3.11/site-packages/smac/model/abstract_model.py", line 152, in train
    return self._train(X, Y)
           ^^^^^^^^^^^^^^^^^
  File "/home/eholden/.pyenv/versions/3.11.3/lib/python3.11/site-packages/smac/model/random_forest/random_forest.py", line 137, in _train
    X = self._impute_inactive(X)
        ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eholden/.pyenv/versions/3.11.3/lib/python3.11/site-packages/smac/model/random_forest/abstract_random_forest.py", line 44, in _impute_inactive
    raise ValueError
ValueError

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[dengdifan]

Hi, could you please check if the error still happens with the new development branch? This issue should be solved by: https://github.com/automl/SMAC3/pull/1065

[charlesjhill]

^ I had the same issue as OP, but I can confirm that #1065 fixes it, thanks!

[benjamc]

Closing this issue as it should be fixed with the new version (#1108 ) which includes this fix #1065 .