ValueError: Output tensors to a Model must be the output of a Keras `Layer` (thus holding past layer metadata). Found: <keras.layers.merge.Concatenate object at 0x7fe49425c5f8>

[yongqianxiao]

Please make sure that this is a Bug or a Feature Request and provide all applicable information asked by the template. If your issue is an implementation question, please ask your question on StackOverflow or on the Keras Slack channel instead of opening a GitHub issue.

System information

• Have I written custom code (as opposed to using example directory):
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): 16.04
• TensorFlow backend (yes / no):
• TensorFlow version: 1.13
• Keras version: 2.2.4
• Python version: 3.6.7
• CUDA/cuDNN version: 10.0
• GPU model and memory: 1080ti

You can obtain the TensorFlow version with: "b'v1.13.1-0-g6612da8951'", 1.13.1' python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
You can obtain the Keras version with: 2.2.4 python -c 'import keras as k; print(k.version)'

Describe the current behavior
the code is from DDPG TORCS

# Now create the model
        self.model, self.weights, self.state = self.create_actor_network(
            state_size, action_size)
        self.target_model, self.target_weights, self.target_state = self.create_actor_network(
            state_size, action_size)
        self.action_gradient = tf.placeholder(tf.float32, [None, action_size])
        self.params_grad = tf.gradients(self.model.output, self.weights,
                                        -self.action_gradient)

def create_actor_network(self, state_size, action_dim):
        print("Now we build the model")
        S = Input(shape=[state_size])
        h0 = Dense(HIDDEN1_UNITS, activation='relu')(S)
        h1 = Dense(HIDDEN2_UNITS, activation='relu')(h0)
        weightInitializer = RandomNormal(mean=0.0, stddev=1e-4)
        Steering = Dense(
            1, activation='tanh', kernel_initializer=weightInitializer)(h1)
        Acceleration = Dense(
            1,
            activation='sigmoid',
            kernel_initializer=weightInitializer)(h1)
        Brake = Dense(
            1,
            activation='sigmoid',
            kernel_initializer=weightInitializer)(h1)
        V = Concatenate([Steering, Acceleration, Brake])
        model = Model(input=S, output=V)
        return model, model.trainable_weights, S

when I run the code, the fellow line raise a error:

 model = Model(input=S, output=V)
ValueError: Output tensors to a Model must be the output of a Keras `Layer` (thus holding past layer metadata). Found: <keras.layers.merge.Concatenate object at 0x7fe49425c5f8>

when I change the line to model=Model(input=S, output=[Steering, Acceleration, Brake]), then that line code is well, but another line code raise a error:

self.params_grad = tf.gradients(self.model.output, self.weights, -self.action_gradient)

  File "/home/nudt302/myPythonDemo/DDPG-Keras-Torcs/ddpg.py", line 176, in <module>
    playGame()
  File "/home/nudt302/myPythonDemo/DDPG-Keras-Torcs/ddpg.py", line 53, in playGame
    actor = ActorNetwork(sess, state_dim, action_dim, BATCH_SIZE, TAU, LRA)
  File "/home/nudt302/myPythonDemo/DDPG-Keras-Torcs/ActorNetwork.py", line 33, in __init__
    -self.action_gradient)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gradients_impl.py", line 664, in gradients
    unconnected_gradients)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gradients_impl.py", line 836, in _GradientsHelper
    gradient_uid)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gradients_impl.py", line 231, in _DefaultGradYs
    raise ValueError("Passed %d grad_ys for %d ys" % (len(grad_ys), len(ys)))
ValueError: Passed 1 grad_ys for 3 ys

can someone help me to solve the problem, it just spend me a whole day. Describe the expected behavior

Code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate the problem.

Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

[littlemountainman]

The model class is a layer object. And your V is obviously not a tensor but it needs a tensor. model=Model(input=S, output=[Steering, Acceleration, Brake]) when you run this, you are giving it a tensor so it works. Your second problem is probably a tensorflow problem just use the same keras gradient function and try to use it again. I will make a pull request to fix your first error.

EDIT: looks like some of the functions of the repo you are trying to build are deprecated.

Best, Leon

[ThomasHenckel]

Hi yongqianxiao

When i look in documentation for the functional API i see "concatenate" not "Concatenate" https://keras.io/getting-started/functional-api-guide/

When i did this change on my network it started training, and the models converge

[aissar]

I also received this error when running the code locally (pulled down from this repo and ran without any modifications). However, the same code works fine when running on Colab.

In order to get this code to run locally I changed the code as follows as it worked. Thank you @ThomasHenckel for identifying this issue.

From this:

concat = layers.Concatenate()([state_out, action_out])

To this:

concat = layers.concatenate([state_out, action_out])