search

automl / ConfigSpace

Domain specific language for configuration spaces in Python. Useful for hyperparameter optimization and algorithm configuration.
https://automl.github.io/ConfigSpace/
Other
202 stars 93 forks source link

Possible bug in ForbiddenEqualsRelation? #389

Open mdorier opened 1 month ago

mdorier commented 1 month ago

I have this sample code:

from ConfigSpace import ConfigurationSpace, ForbiddenEqualsRelation, Integer

cs = ConfigurationSpace()
x =  Integer('x', (0,3))
y =  Integer('y', (0,3))
cs.add(x)
cs.add(y)
cs.add(ForbiddenEqualsRelation(x, y))

which gives me an exception:

ConfigSpace.exceptions.ForbiddenValueError: Given vector violates forbidden clause: Forbidden: x == y

If I'm changing ForbiddenEqualsRelation to ForbiddenGreaterThanRelation the code works fine, generating configurations such that x is always less than or equal to y. I would expect ForbiddenEqualsRelation to allow me to generate configurations such that x is always different from y. Am I misunderstanding the use of ForbiddenEqualsRelation or is there a bug?

eddiebergman commented 1 month ago

Ahh, the issue is actually just a bad error message. The issue you are seeing derives from the fact that the default configuration is forbidden, and ConfigSpace requires the default configuration to be valid.

You can get around this fact by setting a default on one of them not be round((3 - 0)/ 2) == 2

mdorier commented 1 month ago

Thanks, it makes sense, at least it forces the user to provide a valid default configuration, proving that such a configuration exists (otherwise it's a SAT problem).

eddiebergman commented 1 month ago

Good point, from a more practical standpoint, the default is also used internally in configuration neighborhood search, for example, it answers the question that if a conditional was activated that suddenly activated a bunch of new parameters, what should they be set to?