Question regarding the U2-Net architecture

[attozeta]

Hello yingkaisha,

thanks for all the implementations, it really helps people try out different U-Nets.

I tried your U2-Net implementation and ran it with the parametes in your documentation, i.e.

u2net = models.u2net_2d((None, None, 3), n_labels=2, 
                        filter_num_down=[64, 128, 256, 512],
                        activation='ReLU', output_activation='Sigmoid', 
                        batch_norm=True, pool=True, unpool=True, deep_supervision=True, name='u2net')

and it works fine. However I don't get why I get output as follows:

...
    32  1430/1430 [==============================] - 994s 695ms/step - loss: 1.6761
- u2net_output_sup0_activation_loss: 0.2554
- u2net_output_sup1_activation_loss: 0.2537
- u2net_output_sup2_activation_loss: 0.2599
- u2net_output_sup3_activation_loss: 0.2854
- u2net_output_sup4_activation_loss: 0.3481
- u2net_output_final_activation_loss: 0.2735
- u2net_output_sup0_activation_accuracy: 0.9073
- u2net_output_sup1_activation_accuracy: 0.9073
- u2net_output_sup2_activation_accuracy: 0.9047
- u2net_output_sup3_activation_accuracy: 0.8932
- u2net_output_sup4_activation_accuracy: 0.8651
- u2net_output_final_activation_accuracy: 0.9081
- val_loss: 1.7999
- val_u2net_output_sup0_activation_loss: 0.2733
- val_u2net_output_sup1_activation_loss: 0.2743
- val_u2net_output_sup2_activation_loss: 0.2821
- val_u2net_output_sup3_activation_loss: 0.3056
- val_u2net_output_sup4_activation_loss: 0.3655
- val_u2net_output_final_activation_loss: 0.2991
- val_u2net_output_sup0_activation_accuracy: 0.8993
- val_u2net_output_sup1_activation_accuracy: 0.8998
- val_u2net_output_sup2_activation_accuracy: 0.8971
- val_u2net_output_sup3_activation_accuracy: 0.8876
- val_u2net_output_sup4_activation_accuracy: 0.8614
- val_u2net_output_final_activation_accuracy: 0.9002
...

This is just (verbose) reporting from the Keras-API about all the outputs (sup0 to sup4 and final for training and validation), but I am wondering where sup5 is gone as the original paper seems to have 7 outputs and I think you don't add sup5(?) to the output?

At the beginning of the training one gets the following report:

...
The depth of u2net_2d = len(filter_num_down) + len(filter_4f_num) = 6
----------
deep_supervision = True
names of output tensors are listed as follows (the last one is the final output):
    u2net_output_sup0_activation
    u2net_output_sup1_activation
    u2net_output_sup2_activation
    u2net_output_sup3_activation
    u2net_output_sup4_activation
    u2net_output_sup5_activation
    u2net_output_final_activation
,,,

but it seems that sup5 is not in the output stack here. The range goes to L_out-1 so the last output is missing?

So if you find the time I would be happy for an explanation.

Cheers.

[yingkaisha]

You are right. Sorry, I forgot to add the deepest deep-supervision layer. I will have a bug fix later today.

Thank you.

[yingkaisha]

Try pip install keras-unet-collection==0.1.0, the issue should be fixed.

[attozeta]

Wow, that was really fast, thanks. I wasn't sure if it might have been on purpose as the deepest RSU is naturally the hardest to learn, so it might have been on purpose (to not include it into the final loss), however now the implementation is identical to the paper.

So indeed a classic off-by-one error? :)

Thanks, closing.

[yingkaisha]

Thanks for the feedback.

Yeah, I didn't add the deepest tensor for my own use (copied my research code and added print-out information incorrectly).

from keras_unet_collection import base
X_out = base.u2net_2d_base(input_tensor, ...)

This would return all the output tensors and you can create your own model head.