Enable better 'in_channels' exception raise in dynamic model definition.
Description
I briefly list the design below just for a reminder.
Nested layer customization
Logic
Each layer maintain a private attribute self._layers to store possible nested layer. When a Layer is registered as an attribute of another Layer, it is automatically added to the second layer's self._layers. Weights are collected recursively.
Inner layer set to be unaware of LayerNode concept because it does not need to be involved in the model node graph. Inner layer is only regarded as the outer layer‘s 'private' attribute, which is not known by the model.
Example
class MyLayer(tl.layers.Layer):
def __init__(self, name=None):
super(MyLayer, self).__init__(name=name)
self.input_layer = tl.layers.Dense(n_units=20) # it's ok not to pass `in_channels` here
self.build(None)
self._built = True
def build(self, inputs_shape=None):
self.W = self._get_weights('weights', shape=(20, 10))
def forward(self, inputs):
inputs = self.input_layer(inputs)
output = tf.matmul(inputs, self.W)
return output
MyLayer would contain three weights: W of MyLayer, W and b of the inner Dense.
'in_channels' exception in dynamic model
Now it is checked when setting Layer as an attribute of dynamic Model. If in_channels not provided, there will be an exception raised, e.g.
The registered layer `dense_4` should be built in advance. Do you forget to pass the keyword argument 'in_channels'?
Checklist
Motivation and Context
Description
I briefly list the design below just for a reminder.
Nested layer customization
self._layers
to store possible nested layer. When aLayer
is registered as an attribute of anotherLayer
, it is automatically added to the second layer'sself._layers
. Weights are collected recursively. Inner layer set to be unaware ofLayerNode
concept because it does not need to be involved in the model node graph. Inner layer is only regarded as the outer layer‘s 'private' attribute, which is not known by the model.Example
MyLayer
would contain three weights:W
ofMyLayer
,W
andb
of the innerDense
.Layer
as an attribute of dynamicModel
. Ifin_channels
not provided, there will be an exception raised, e.g.