automl / ConfigSpace

Domain specific language for configuration spaces in Python/Cython. Useful for hyperparameter optimization and algorithm configuration.
https://automl.github.io/ConfigSpace/
Other
193 stars 89 forks source link

`HyperparameterNotFoundError` prohibits `.get` from functioning as expected #348

Open PGijsbers opened 5 months ago

PGijsbers commented 5 months ago

I am not very familiar with the project, but I came upon this and it looks to me like an oversight (given the refactor of search space to behave more dict-like).

The use of the custom HyperparameterNotFoundError instead of a KeyError breaks the inherited logic of get:

816         def get(self, key, default=None):
817             'D.get(k[,d]) -> D[k] if k in D, else d.  d defaults to None.'
818             try:
819                 return self[key]
820             except KeyError:
821                 return default

Since line 819 fails with a HyperparameterNotFoundError, the .get call now fails if the key is missing from the search space. I recon the easiest would be to derive the custom error from a KeyError, but I don't know what other implications this would have for the project.

MWE:

from ConfigSpace import ConfigurationSpace

cs = ConfigurationSpace(
    name="myspace",
    space={
        "a": (0.1, 1.5),  # UniformFloat
        "b": (2, 10),  # UniformInt
        "c": ["mouse", "cat", "dog"],  # Categorical
    },
)

cs.get("a")  # Correctly gets the UniformFloat
cs.get("d")  # gets output below
>>> cs.get("d")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/_collections_abc.py", line 819, in get
    return self[key]
  File "/Users/pietergijsbers/repositories/gama/venv/lib/python3.10/site-packages/ConfigSpace/configuration_space.py", line 1139, in __getitem__
    raise HyperparameterNotFoundError(key, space=self)
ConfigSpace.exceptions.HyperparameterNotFoundError: Hyperparameter d not found in space.
Configuration space object:
  Hyperparameters:
myspace
    a, Type: UniformFloat, Range: [0.1, 1.5], Default: 0.8
    b, Type: UniformInteger, Range: [2, 10], Default: 6
    c, Type: Categorical, Choices: {mouse, cat, dog}, Default: mouse

workaround

dict(cs).get("d")
eddiebergman commented 2 months ago

Fixed in #346 which did a major re-work of many things. Hopefully released next week!

If you can run the GAMA test sweet with that new branch, that would be actually very helpful for finding any issues!

If you depend on SMAC in anyway, I've also confirmed that branch passes all SMAC unittests.