Randomized Seach CV is not working with KerasRegressor

I have this function which build and compiles the model:

def build_model(n_hidden=1, n_neurons=30, learning_rate=3e-3, input_shape=[8]):
    model = keras.models.Sequential()
    model.add(keras.layers.InputLayer(input_shape=input_shape))
    for layer in range(n_hidden):
        model.add(keras.layers.Dense(n_neurons, activation="relu"))
    model.add(keras.layers.Dense(1))
    optimizer = keras.optimizers.SGD(learning_rate=learning_rate)
    model.compile(loss="mse", optimizer=optimizer)
    return model

And here I am passing that model to KerasRegressor

from scikeras.wrappers import KerasRegressor
keras_reg = KerasRegressor(build_model)

Here's the code for RandomizedSearchCV:

from scipy.stats import reciprocal
from sklearn.model_selection import RandomizedSearchCV

param_distribs = {
    "n_hidden": [0, 1, 2, 3],
    "n_neurons": np.arange(1, 100).tolist(),
    "learning_rate": reciprocal(3e-4, 3e-2).rvs(1000).tolist(),
}

rnd_search_cv = RandomizedSearchCV(keras_reg, param_distribs, n_iter=10, cv=3, verbose=2)
rnd_search_cv.fit(X_train, y_train, epochs=100,
                  validation_data=(X_valid, y_valid),
                  callbacks=[keras.callbacks.EarlyStopping(patience=10)])

This is throwing following error:

Fitting 3 folds for each of 10 candidates, totalling 30 fits
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[51], line 11
      4 param_distribs = {
      5     "n_hidden": [0, 1, 2, 3],
      6     "n_neurons": np.arange(1, 100).tolist(),
      7     "learning_rate": reciprocal(3e-4, 3e-2).rvs(1000).tolist(),
      8 }
     10 rnd_search_cv = RandomizedSearchCV(keras_reg, param_distribs, n_iter=10, cv=3, verbose=2)
---> 11 rnd_search_cv.fit(X_train, y_train, epochs=100,
     12                   validation_data=(X_valid, y_valid),
     13                   callbacks=[keras.callbacks.EarlyStopping(patience=10)])

<<Few more lines of error >>>

File ~/anaconda3/envs/latest/lib/python3.11/site-packages/sklearn/model_selection/_validation.py:720, in _fit_and_score(estimator, X, y, scorer, train, test, verbose, parameters, fit_params, return_train_score, return_parameters, return_n_test_samples, return_times, return_estimator, split_progress, candidate_progress, error_score)
    717     for k, v in parameters.items():
    718         cloned_parameters[k] = clone(v, safe=False)
--> 720     estimator = estimator.set_params(**cloned_parameters)
    722 start_time = time.time()
    724 X_train, y_train = _safe_split(estimator, X, y, train)

File ~/anaconda3/envs/latest/lib/python3.11/site-packages/scikeras/wrappers.py:1165, in BaseWrapper.set_params(self, **params)
   1161             super().set_params(**{param: value})
   1162         except ValueError:
   1163             # Give a SciKeras specific user message to aid
   1164             # in moving from the Keras wrappers
-> 1165             raise ValueError(
   1166                 f"Invalid parameter {param} for estimator {self.__name__}."
   1167                 "\nThis issue can likely be resolved by setting this parameter"
   1168                 f" in the {self.__name__} constructor:"
   1169                 f"\n`{self.__name__}({param}={value})`"
   1170                 "\nCheck the list of available parameters with"
   1171                 " `estimator.get_params().keys()`"
   1172             ) from None
   1173 return self

ValueError: Invalid parameter n_neurons for estimator KerasRegressor.
This issue can likely be resolved by setting this parameter in the KerasRegressor constructor:
`KerasRegressor(n_neurons=77)`
Check the list of available parameters with `estimator.get_params().keys()`

Any way to solve to make it working??

NOTE: I have omitted few stack trace from error as it was getting too large. If needed let me know, I will provide whole stack trace.

Like the error says you need to include the parameter in the call to the KerasRegressor constructor. You can also get SciKeras to dynamically give you the input shape

import numpy as np
from sklearn.model_selection import RandomizedSearchCV
from scikeras.wrappers import KerasRegressor
from scipy.stats import reciprocal
from tensorflow import keras

def build_model(n_hidden, n_neurons, learning_rate, meta):
    model = keras.models.Sequential()
    model.add(keras.layers.InputLayer(input_shape=(meta['n_features_in_'],)))
    for layer in range(n_hidden):
        model.add(keras.layers.Dense(n_neurons, activation="relu"))
    model.add(keras.layers.Dense(1))
    optimizer = keras.optimizers.SGD(learning_rate=learning_rate)
    model.compile(loss="mse", optimizer=optimizer)
    return model

keras_reg = KerasRegressor(build_model, n_hidden=1, n_neurons=30, learning_rate=3e-3)

param_distribs = {
    "n_hidden": [0, 1, 2, 3],
    "n_neurons": np.arange(1, 100).tolist(),
    "learning_rate": reciprocal(3e-4, 3e-2).rvs(1000).tolist(),
}

x = np.random.uniform(size=(10, 8))
y = np.random.randint(0, 1, size=(10,))

rnd_search_cv = RandomizedSearchCV(keras_reg, param_distribs, n_iter=1, cv=1, verbose=2)
rnd_search_cv.fit(x, y, epochs=100, callbacks=[keras.callbacks.EarlyStopping(patience=10)])

It's probably worth taking a look at these two tutorials:

adriangb / scikeras

Randomized Seach CV is not working with KerasRegressor #312