Unused kernel regularizers during training

[klekass]

In the model l2 kernel regularizers are defined

conv = tf.keras.layers.Conv2D(filters=filters_shape[-1], 
        kernel_size = filters_shape[0], 
        strides=strides, padding=padding,
        use_bias=not bn, 
        kernel_regularizer=tf.keras.regularizers.l2(0.0005),
        kernel_initializer=tf.random_normal_initializer(stddev=0.01),
        bias_initializer=tf.constant_initializer(0.))(input_layer)

However, during training, loss is computed manually using gradient tape (instead of using keras' model.fit() function) with the following three loss values:

total_loss = giou_loss + conf_loss + prob_loss

Is it possible that we are missing regularizer loss here? I tested it by setting kernel_regularizer=None which resulted in the exact same total loss. I suggest manually adding the regularizer loss using:

def regularizer_loss(model):
    """Retrieve kernel regularizer loss from layers with kernel regularizer"""
    loss = 0
    for layer in model.layers:
        if hasattr(layer, "kernel_regularizer") and layer.kernel_regularizer:
            loss += layer.kernel_regularizer(layer.kernel)
    return loss

total_loss = giou_loss + conf_loss + prob_loss + regularizer_loss(model)