Bayesian Tuner cannot handle NaNs as eval results

[YevhenAkimov]

Hi, during optimization with the bayesian tuner I encountered an error:

Trial 18 Complete [00h 00m 37s]
val_loss: nan
Best val_loss So Far: 0.00023380108177661896
Total elapsed time: 00h 11m 41s
/usr/local/lib/python3.6/dist-packages/kerastuner/engine/metrics_tracking.py:92: RuntimeWarning: All-NaN axis encountered
  return np.nanmin(values)
Traceback (most recent call last):
  File "GCP_OPT_C1_G_Att_D.py", line 387, in <module>
    tuner.search(model.train_X,model.train_Y,epochs=hypers['OPT']['epochs'],validation_split=hypers['OPT']['validation_split'],callbacks=[ stop_early , RLR ])
  File "/usr/local/lib/python3.6/dist-packages/kerastuner/engine/base_tuner.py", line 121, in search
    trial = self.oracle.create_trial(self.tuner_id)
  File "/usr/local/lib/python3.6/dist-packages/kerastuner/engine/oracle.py", line 165, in create_trial
    response = self._populate_space(trial_id)
  File "/usr/local/lib/python3.6/dist-packages/kerastuner/tuners/bayesian.py", line 112, in _populate_space
    raise e
  File "/usr/local/lib/python3.6/dist-packages/kerastuner/tuners/bayesian.py", line 105, in _populate_space
    self.gpr.fit(x, y)
  File "/usr/local/lib/python3.6/dist-packages/sklearn/gaussian_process/_gpr.py", line 193, in fit
    ensure_2d=True, dtype="numeric")
  File "/usr/local/lib/python3.6/dist-packages/sklearn/base.py", line 433, in _validate_data
    X, y = check_X_y(X, y, **check_params)
  File "/usr/local/lib/python3.6/dist-packages/sklearn/utils/validation.py", line 63, in inner_f
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/sklearn/utils/validation.py", line 824, in check_X_y
    ensure_2d=False, dtype=None)
  File "/usr/local/lib/python3.6/dist-packages/sklearn/utils/validation.py", line 63, in inner_f
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/sklearn/utils/validation.py", line 664, in check_array
    allow_nan=force_all_finite == 'allow-nan')
  File "/usr/local/lib/python3.6/dist-packages/sklearn/utils/validation.py", line 106, in _assert_all_finite
    msg_dtype if msg_dtype is not None else X.dtype)
ValueError: Input contains NaN, infinity or a value too large for dtype('float64').`

As far as I understand this is due to the difference in the error messages produced by scipy versions (I have scipy 1.16.1). Currently, Keras-tuner captures: except ValueError as e: if "array must not contain infs or NaNs" in str(e): return self._random_populate_space() raise e which is different from: ValueError: Input contains NaN, infinity or a value too large for dtype('float64').

So, I guess the fix should be straightforward. Thanks.

[haifeng-jin]

This bug is fixed. You can verify with the following code snippet:

import keras_tuner as kt

class MyTuner(kt.BayesianOptimization):
    def run_trial(self, trial, **kwargs):
        if trial.hyperparameters.Boolean("break"):
            return float("nan")
        return trial.hyperparameters.Float("result", 0, 1)

tuner = MyTuner(max_trials=10, overwrite=True, directory="test")
tuner.search()
tuner.results_summary()

Or use this notebook.

More details: From the original context, the BO tuner is using sklearn, which is no longer the case. The Tuner base class can handle NaNs.

Please reopen it if it is not fixed.