search

JanMarcelKezmann / TensorFlow-Advanced-Segmentation-Models

A Python Library for High-Level Semantic Segmentation Models based on TensorFlow and Keras with pretrained backbones.
Other
151 stars 46 forks source link

UNet Model has too many non-trainable variables #28

Open TillBeemelmanns opened 1 year ago

TillBeemelmanns commented 1 year ago

I was testing the UNet model along with a mobilenetv2 and saw that there are a lot non-trainable params, which I could not explain. I quick lookup in the model revealed that besides the non-trainable BN params, there are also normal kernels that are marked as non-trainable. I guess this behavior is not desired.

Correct me if I am wrong, but I guess the default trainable parameter should be set to true ? https://github.com/JanMarcelKezmann/TensorFlow-Advanced-Segmentation-Models/blob/c06d3d288f0f445fc4c8f567cd2960494a6d70ff/tensorflow_advanced_segmentation_models/models/_custom_layers_and_blocks.py#L214-L219

Model: "UNet_mobilenetv2_1.00_None"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input (InputLayer)           [(None, None, None, 3)]   0         
_________________________________________________________________
u_net (UNet)                 (None, None, None, 30)    15387454  
=================================================================
Total params: 15,387,454
Trainable params: 2,384,222
Non-trainable params: 13,003,232
for variable in model.non_trainable_variables:
    print(variable.name)
u_net/convolution_bn_activation/batch_normalization/moving_mean:0
u_net/convolution_bn_activation/batch_normalization/moving_variance:0
u_net/convolution_bn_activation_1/batch_normalization_1/moving_mean:0
u_net/convolution_bn_activation_1/batch_normalization_1/moving_variance:0
u_net/upsample_x2__block/conv2d/kernel:0
u_net/upsample_x2__block/conv2d/bias:0
u_net/upsample_x2__block/convolution_bn_activation_2/conv2d_2/kernel:0
u_net/upsample_x2__block/convolution_bn_activation_2/conv2d_2/bias:0
u_net/upsample_x2__block/convolution_bn_activation_2/batch_normalization_2/gamma:0
u_net/upsample_x2__block/convolution_bn_activation_2/batch_normalization_2/beta:0
u_net/upsample_x2__block/convolution_bn_activation_2/batch_normalization_2/moving_mean:0
u_net/upsample_x2__block/convolution_bn_activation_2/batch_normalization_2/moving_variance:0
u_net/upsample_x2__block/convolution_bn_activation_3/conv2d_3/kernel:0
u_net/upsample_x2__block/convolution_bn_activation_3/conv2d_3/bias:0
u_net/upsample_x2__block/convolution_bn_activation_3/batch_normalization_3/gamma:0
u_net/upsample_x2__block/convolution_bn_activation_3/batch_normalization_3/beta:0
u_net/upsample_x2__block/convolution_bn_activation_3/batch_normalization_3/moving_mean:0
u_net/upsample_x2__block/convolution_bn_activation_3/batch_normalization_3/moving_variance:0
u_net/upsample_x2__block_1/conv2d_1/kernel:0
u_net/upsample_x2__block_1/conv2d_1/bias:0
u_net/upsample_x2__block_1/convolution_bn_activation_4/conv2d_4/kernel:0
u_net/upsample_x2__block_1/convolution_bn_activation_4/conv2d_4/bias:0
u_net/upsample_x2__block_1/convolution_bn_activation_4/batch_normalization_4/gamma:0
u_net/upsample_x2__block_1/convolution_bn_activation_4/batch_normalization_4/beta:0
u_net/upsample_x2__block_1/convolution_bn_activation_4/batch_normalization_4/moving_mean:0
u_net/upsample_x2__block_1/convolution_bn_activation_4/batch_normalization_4/moving_variance:0
u_net/upsample_x2__block_1/convolution_bn_activation_5/conv2d_5/kernel:0
u_net/upsample_x2__block_1/convolution_bn_activation_5/conv2d_5/bias:0
u_net/upsample_x2__block_1/convolution_bn_activation_5/batch_normalization_5/gamma:0
u_net/upsample_x2__block_1/convolution_bn_activation_5/batch_normalization_5/beta:0
u_net/upsample_x2__block_1/convolution_bn_activation_5/batch_normalization_5/moving_mean:0
u_net/upsample_x2__block_1/convolution_bn_activation_5/batch_normalization_5/moving_variance:0
u_net/upsample_x2__block_2/conv2d_2/kernel:0
u_net/upsample_x2__block_2/conv2d_2/bias:0
u_net/upsample_x2__block_2/convolution_bn_activation_6/conv2d_6/kernel:0
u_net/upsample_x2__block_2/convolution_bn_activation_6/conv2d_6/bias:0
u_net/upsample_x2__block_2/convolution_bn_activation_6/batch_normalization_6/gamma:0
u_net/upsample_x2__block_2/convolution_bn_activation_6/batch_normalization_6/beta:0
u_net/upsample_x2__block_2/convolution_bn_activation_6/batch_normalization_6/moving_mean:0
u_net/upsample_x2__block_2/convolution_bn_activation_6/batch_normalization_6/moving_variance:0
u_net/upsample_x2__block_2/convolution_bn_activation_7/conv2d_7/kernel:0
u_net/upsample_x2__block_2/convolution_bn_activation_7/conv2d_7/bias:0
u_net/upsample_x2__block_2/convolution_bn_activation_7/batch_normalization_7/gamma:0
u_net/upsample_x2__block_2/convolution_bn_activation_7/batch_normalization_7/beta:0
u_net/upsample_x2__block_2/convolution_bn_activation_7/batch_normalization_7/moving_mean:0
u_net/upsample_x2__block_2/convolution_bn_activation_7/batch_normalization_7/moving_variance:0
u_net/upsample_x2__block_3/conv2d_3/kernel:0
u_net/upsample_x2__block_3/conv2d_3/bias:0
u_net/upsample_x2__block_3/convolution_bn_activation_8/conv2d_8/kernel:0
u_net/upsample_x2__block_3/convolution_bn_activation_8/conv2d_8/bias:0
u_net/upsample_x2__block_3/convolution_bn_activation_8/batch_normalization_8/gamma:0
u_net/upsample_x2__block_3/convolution_bn_activation_8/batch_normalization_8/beta:0
u_net/upsample_x2__block_3/convolution_bn_activation_8/batch_normalization_8/moving_mean:0
u_net/upsample_x2__block_3/convolution_bn_activation_8/batch_normalization_8/moving_variance:0
u_net/upsample_x2__block_3/convolution_bn_activation_9/conv2d_9/kernel:0
u_net/upsample_x2__block_3/convolution_bn_activation_9/conv2d_9/bias:0
u_net/upsample_x2__block_3/convolution_bn_activation_9/batch_normalization_9/gamma:0
u_net/upsample_x2__block_3/convolution_bn_activation_9/batch_normalization_9/beta:0
u_net/upsample_x2__block_3/convolution_bn_activation_9/batch_normalization_9/moving_mean:0
u_net/upsample_x2__block_3/convolution_bn_activation_9/batch_normalization_9/moving_variance:0
u_net/upsample_x2__block_4/conv2d_4/kernel:0
u_net/upsample_x2__block_4/conv2d_4/bias:0
u_net/upsample_x2__block_4/convolution_bn_activation_10/conv2d_10/kernel:0
u_net/upsample_x2__block_4/convolution_bn_activation_10/conv2d_10/bias:0
u_net/upsample_x2__block_4/convolution_bn_activation_10/batch_normalization_10/gamma:0
u_net/upsample_x2__block_4/convolution_bn_activation_10/batch_normalization_10/beta:0
u_net/upsample_x2__block_4/convolution_bn_activation_10/batch_normalization_10/moving_mean:0
u_net/upsample_x2__block_4/convolution_bn_activation_10/batch_normalization_10/moving_variance:0
u_net/upsample_x2__block_4/convolution_bn_activation_11/conv2d_11/kernel:0
u_net/upsample_x2__block_4/convolution_bn_activation_11/conv2d_11/bias:0
u_net/upsample_x2__block_4/convolution_bn_activation_11/batch_normalization_11/gamma:0
u_net/upsample_x2__block_4/convolution_bn_activation_11/batch_normalization_11/beta:0
u_net/upsample_x2__block_4/convolution_bn_activation_11/batch_normalization_11/moving_mean:0
u_net/upsample_x2__block_4/convolution_bn_activation_11/batch_normalization_11/moving_variance:0