Yelp / MOE

A global, black box optimization engine for real world metric optimization.
Other
1.31k stars 140 forks source link

[Python] Improve hyperparameter validation in the REST interface #216

Open suntzu86 opened 10 years ago

suntzu86 commented 10 years ago

As of #214, Python (REST interface, via colander) simply checks that all hyperparameters are positive floats. This is true for current covariance functions, but it need not always be true.

Ideally, covariance functions should expose a hyperparameter_validate() function (or similar), which we then call from colander (or somewhere in the REST pipeline) to check hyperparameters.

suntzu86 commented 10 years ago

http://colander.readthedocs.org/en/latest/api.html#validators shows how to add a "Function" validator where colander will pass the data to any function

and http://colander.readthedocs.org/en/latest/basics.html#subclassing-schemanode shows how to create fancier custom SchemaNode

one or both of these will do what we want, I think.

suntzu86 commented 10 years ago

Also kind of stumbled on this in addition to colander.Function validator, you can:

def positive_validator(node, cstruct):
    """Raise an exception if the node value (cstruct) is non-positive or non-finite.

    :param node: the node being validated
    :type node: colander.SchemaNode subclass instance
    :param cstruct: the value being validated
    :type cstruct: float
    :raise: colander.Invalid if cstruct value is bad

    """
    if not 0.0 < cstruct < float('inf'):
        raise colander.Invalid(node, msg='Value must be positive and finite.')

positive_float = colander.SchemaNode(
    colander.Float(),
    validator=positive_validator,
    title='Positive Float',
    default=None,
)

so the validator field can be set to anything that takes node, value as in the argument list. There's possibly/probably more to it than that but this does work too.

suntzu86 commented 10 years ago

303 or #34 will probably help with this. We could set up colander's schema binding mechanic to connect the schemas to the right validator after the type is established. Or we could hook up de/serialization to the actual CovarianceInterface implementations directly.