Support Non-Linear Step Size for Int and Float Hyperparameter Types

[campellcl]

Is your feature request related to a problem? Please describe. Sometimes it may make sense to specify a non-linear range of hyperparameters. For instance, we may wish to experiment with a particular hyperparameter containing as values the powers of two: (i.e. 2^0, 2^1, 2^2). It appears that this is currently difficult to do in keras-tuner, as the step argument in the hyperparameters.Int constructor only takes an integer value as input. The same is true for the hyperparameters.Float constructor.

Describe the solution you'd like It would be very helpful to be able to specify a Callable as an argument tostep, for instance:

num_units: int = HyperParameters.Int(name='num_units', min_value=0, max_value=6, step=lambda x: 2**x, sampling=None, default=None, parent_name=None, parent_values=None)

Or perhaps, allow a pre-defined list of steps, as in:

steps: List[int] = [2**i for i in range(6)]
num_units: int = HyperParameters.Int(name='num_units', min_value=2**0, max_value=2**6, step=steps, sampling=None, default=None, parent_name=None, parent_values=None)

Describe alternatives you've considered It is mentioned in the documentation for the step argument of the Float constructor that:

... some Oracles can infer an optimal step automatically.

I have looked at the Oracle base class documentation, it may be possible to implement this in the populate_space method, but it would be misleading. As we would be defining a step size up front as an integer, and then ignoring that information when populating the search space for each trial.

Additional context None.

[Cheebi]

Probably too late, but isn't the sampling parameter what you are looking for?

[sayantanghosh]

@campellcl

I am also probably too late, but I wanted to do something similar with the number of dense units, and this solution worked for me: hp.num_neurons = hp.Choice('num_neurons', [2 ** _i for _i in range(8)])

Hope this helps you, or anyone else who comes across this question.