Classifier ValueError: Input 0 of layer dense is incompatible with the layer: expected axis -1 of input shape to have value 1 but received input with shape (1, 1024)

[rgap]

Description

Hi @ltiao,

I'm trying to run the example (master branch). Please, could you let me know what I'm doing wrong here. I'm getting the error:

WARNING:tensorflow:Model was constructed with shape (None, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 1), dtype=tf.float32, name='dense_input'), name='dense_input', description="created by layer 'dense_input'"), but it was called on an input with incompatible shape (1, 1024).
Traceback (most recent call last):
  File "/Users/rgap/rgap_mlib/quickmlexperiments/global_opt/surrogate_based/BORE/bore_test.py", line 56, in <module>
    x_next = classifier.argmax(method="L-BFGS-B", num_starts=3, bounds=bounds)
  File "/Users/rgap/Libraries/bore/bore/mixins.py", line 33, in argmax
    for i, res in enumerate(self.maxima(bounds, *args, **kwargs)):
  File "/Users/rgap/Libraries/bore/bore/mixins.py", line 19, in maxima
    return minimize_multi_start(self._func_min, bounds=bounds,
  File "/Users/rgap/Libraries/bore/bore/optimizers/base.py", line 52, in new_minimizer
    values, _ = fn(X_init)
  File "/Users/rgap/Libraries/bore/bore/decorators.py", line 75, in new_fn
    outputs = fn(*new_args)
  File "/Users/rgap/.virtualenvs/rgap-lectures/lib/python3.9/site-packages/tensorflow/python/eager/def_function.py", line 889, in __call__
    result = self._call(*args, **kwds)
  File "/Users/rgap/.virtualenvs/rgap-lectures/lib/python3.9/site-packages/tensorflow/python/eager/def_function.py", line 933, in _call
    self._initialize(args, kwds, add_initializers_to=initializers)
  File "/Users/rgap/.virtualenvs/rgap-lectures/lib/python3.9/site-packages/tensorflow/python/eager/def_function.py", line 763, in _initialize
    self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access
  File "/Users/rgap/.virtualenvs/rgap-lectures/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 3050, in _get_concrete_function_internal_garbage_collected
    graph_function, _ = self._maybe_define_function(args, kwargs)
  File "/Users/rgap/.virtualenvs/rgap-lectures/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 3444, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/Users/rgap/.virtualenvs/rgap-lectures/lib/python3.9/site-packages/tensorflow/python/eager/function.py", line 3279, in _create_graph_function
    func_graph_module.func_graph_from_py_func(
  File "/Users/rgap/.virtualenvs/rgap-lectures/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py", line 999, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/Users/rgap/.virtualenvs/rgap-lectures/lib/python3.9/site-packages/tensorflow/python/eager/def_function.py", line 672, in wrapped_fn
    out = weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "/Users/rgap/.virtualenvs/rgap-lectures/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py", line 986, in wrapper
    raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:

    /Users/rgap/Libraries/bore/bore/decorators.py:60 value_and_gradient_fn  *
        val = value_fn(x)
    /Users/rgap/Libraries/bore/bore/decorators.py:37 new_fn  *
        return tf.squeeze(fn(*args, **kwargs), axis=axis)
    /Users/rgap/Libraries/bore/bore/decorators.py:38 new_fn  *
        batch_output = fn(batch_input)
    /Users/rgap/Libraries/bore/bore/base.py:40 fn  *
        return transform(model(x))
    /Users/rgap/.virtualenvs/rgap-lectures/lib/python3.9/site-packages/tensorflow/python/keras/engine/base_layer.py:1030 __call__  **
        outputs = call_fn(inputs, *args, **kwargs)
    /Users/rgap/.virtualenvs/rgap-lectures/lib/python3.9/site-packages/tensorflow/python/keras/engine/sequential.py:380 call  **
        return super(Sequential, self).call(inputs, training=training, mask=mask)
    /Users/rgap/.virtualenvs/rgap-lectures/lib/python3.9/site-packages/tensorflow/python/keras/engine/functional.py:420 call
        return self._run_internal_graph(
    /Users/rgap/.virtualenvs/rgap-lectures/lib/python3.9/site-packages/tensorflow/python/keras/engine/functional.py:556 _run_internal_graph
        outputs = node.layer(*args, **kwargs)
    /Users/rgap/.virtualenvs/rgap-lectures/lib/python3.9/site-packages/tensorflow/python/keras/engine/base_layer.py:1013 __call__
        input_spec.assert_input_compatibility(self.input_spec, inputs, self.name)
    /Users/rgap/.virtualenvs/rgap-lectures/lib/python3.9/site-packages/tensorflow/python/keras/engine/input_spec.py:251 assert_input_compatibility
        raise ValueError(

    ValueError: Input 0 of layer dense is incompatible with the layer: expected axis -1 of input shape to have value 1 but received input with shape (1, 1024)

Code

import numpy as np
from bore.models import MaximizableSequential
from tensorflow.keras.layers import Dense

class Forrester():

    def cost(self, x):
        """ Cost function. """
        x = np.array(x)
        # Cost
        c = np.zeros(x.shape[1:])
        # Calculate Cost
        c = np.sin(12*x[0] - 4)*(6*x[0] - 2)**2
        # Return Cost
        return c

# build model
classifier = MaximizableSequential()
classifier.add(Dense(16, activation="relu"))
classifier.add(Dense(16, activation="relu"))
classifier.add(Dense(1, activation="sigmoid"))

# compile model
classifier.compile(optimizer="adam", loss="binary_crossentropy")

bounds = [(0, 1)]
f = Forrester()
features_init = [0.025, 0.15, 0.975, 0.9]
targets_init = [f.cost([x]) for x in features_init]

num_iterations = 20

features = []
targets = []

# initial design
features.extend(features_init)
targets.extend(targets_init)

for i in range(num_iterations):

    # construct classification problem
    X = np.vstack(features)
    y = np.hstack(targets)

    tau = np.quantile(y, q=0.25)
    z = np.less(y, tau)

    # update classifier
    classifier.fit(X, z, epochs=200, batch_size=64, verbose=0)

    # suggest new candidate
    x_next = classifier.argmax(method="L-BFGS-B", num_starts=3, bounds=bounds)

    # evaluate blackbox function
    y_next = f.cost(x_next)

    # update dataset
    features.append(x_next)
    targets.append(y_next)

[ltiao]

Thanks for reporting this issue! Could you please report what the shape of X and y are? I suspect you might be applying np.vstack to a 1D array.

[rgap]

The shapes

type(X) = <class 'numpy.ndarray'> X.shape = (4, 1)

type(y) = <class 'numpy.ndarray'> y.shape = (4,)

z.shape = (4,)

[ltiao]

And which line is raising the ValueError exception?

[ltiao]

Never mind, I can see it in the trace you posted above. Let me look into it and get back to you soon

[rgap]

Thanks, and it is in your code when converting to a tensor I think

new_args = map(tf.convert_to_tensor, args) outputs = fn(*new_args)

[ltiao]

This seems to only be a problem with 1D functions. I will issue a patch soon. Feel free to try it out with functions of several variables in the meantime :)

[ltiao]

Fixed in #8