Build keras model

[Ynjxsjmh]

In Specifying the input shape in advance | TensorFlow Core, it says:

Generally, all layers in Keras need to know the shape of their inputs in order to be able to create their weights.

It then gives two ways to specify the shape of inputs:

1. Start your model by passing an Input object to your model, so that it knows its input shape from the start:

model = keras.Sequential()
model.add(keras.Input(shape=(4,)))
model.add(layers.Dense(2, activation="relu"))

2. A simple alternative is to just pass an input_shape argument to your first layer:

model = keras.Sequential()
model.add(layers.Dense(2, activation="relu", input_shape=(4,)))

But I haven't seen input shapes defined in your code:

(Reinforce)Policy Gradient with TensorFlow2.x

class model(tf.keras.Model):
  def __init__(self):
    super().__init__()
    self.d1 = tf.keras.layers.Dense(30,activation='relu')
    self.d2 = tf.keras.layers.Dense(30,activation='relu')
    self.out = tf.keras.layers.Dense(env.action_space.n,activation='softmax')

  def call(self, input_data):
    x = tf.convert_to_tensor(input_data)
    x = self.d1(x)
    x = self.d2(x)
    x = self.out(x)
    return x

Proximal Policy Optimization (PPO) With TensorFlow 2.x

class critic(tf.keras.Model):
  def __init__(self):
    super().__init__()
    self.d1 = tf.keras.layers.Dense(128,activation='relu')
    self.v = tf.keras.layers.Dense(1, activation = None)

  def call(self, input_data):
    x = self.d1(input_data)
    v = self.v(x)
    return v

class actor(tf.keras.Model):
  def __init__(self):
    super().__init__()
    self.d1 = tf.keras.layers.Dense(128,activation='relu')
    self.a = tf.keras.layers.Dense(2,activation='softmax')

  def call(self, input_data):
    x = self.d1(input_data)
    a = self.a(x)
    return a

You just call self.d1(input_data) directly, does tensorflow detect the input shape implicitly?

Besides, in your PG code you did tf.convert_to_tensor(input_data) before calling self.d1(x), while you didn't do that in PPO code. Does this mean tf.convert_to_tensor() is unnecessary?

[abhisheksuran]

1. TensorFlow models can be made using Sequential API, Functional API, and Subclassing. Here, I have used subclassing, and you do not need to specify input shape explicitly in that.
2. tf.convert_to_tensor(input_data) is optional here.

[Ynjxsjmh]

Thanks for your reply, it helps a lot. I also find a related question on SO: Model subclassing : no input dimension - Stack Overflow.