Question: Can we add weights for a Categorical Hyperparameter?

[andynader]

Description

Can I add weights to the different choices in a CategoricalHyperparameter?

The API documentation states that this is possible: https://automl.github.io/ConfigSpace/master/API-Doc.html#categorical-hyperparameters

But when I try to do this through:

network_precision_weights=[0.1,0.9]

quantize_bool = CategoricalHyperparameter("quantize_bool", choices=["0", "1"], weights=network_precision_weights)

I get an error:

ValueError: The pcs format does not support categorical hyperparameters with assigend weights/probabilities (for hyperparameter quantize_bool)

[benjamc]

Hi andynader,

this is an issue of ConfigSpace, which version of ConfigSpace do you use? Maybe you need to update ConfigSpace.

For me your/the following commands work for ConfigSpace version 0.4.19.

from ConfigSpace import CategoricalHyperparameter
network_precision_weights=[0.1,0.9]
quantize_bool = CategoricalHyperparameter("quantize_bool", choices=["0", "1"], weights=network_precision_weights)

If you have any more issues, would you open an issue at ConfigSpace (https://github.com/automl/ConfigSpace/issues)?

[andynader]

Hi @benjamc,

It looks like the issue is with SMAC4HPO, not ConfigSpace. Here's a simple toy example to reproduce the ValueError given in my first comment. Perhaps SMAC4HPO doesn't support weights?

from smac.facade.smac_hpo_facade import SMAC4HPO from smac.scenario.scenario import Scenario from ConfigSpace import ConfigurationSpace from ConfigSpace.hyperparameters import CategoricalHyperparameter import numpy as np

cs=ConfigurationSpace() network_precision_weights=[0.1,0.9] quantize_bool = CategoricalHyperparameter("quantize_bool", choices=["0", "1"], weights=network_precision_weights) cs.add_hyperparameters([quantize_bool])

fitness_fun=lambda x:1

scenario = Scenario({"run_obj": "quality", "runcount-limit": 128,
"cs": cs,
"deterministic": "false"})

If I run the above code only, I don't get an error. The error only occurs after the following line:

smac = SMAC4HPO(scenario=scenario, rng=np.random.RandomState(42), tae_runner=fitness_fun)

[renesass]

Hi,

try the following: smac/utils/io/output_writer.py and in the method save_configspace add the following in the beginning:

if output_format == "pcs_new":
            return

That means, we simply don't save in pcs format (which is in the most cases not needed anyway). Don't forget to pip install . if you didn't use pip install -e .. I will close this issue but feel free to open it again. :)

Best, René