The pretrained weights and the network architecture of Pytorch version

[vic85821]

Hi Zongwei,

Thanks for the release of the Pytorch version.

However, there is some difference between the weights of Keras and Pytorch. The network utilizes the UpSampling3D as the up convolution layer in the Keras version, therefore, there is no trainable weight in those layers. In contrast, the Pytorch model adopts the ConvTranspose3d, which contains trainable weights.

Which version is recommended for us if we would like to reproduce the excellent performance in your accepted paper?

Best, Yu-Cheng

[MrGiovanni]

Hi Yu-Cheng,

Based on the feedback, I heard that the one implemented in PyTorch may achieve slightly better performance, but we do NOT try to reach a decisive conclusion on which deep architectures work the best in medical imaging or computer vision. From time to time, different deep model architectures are invented every year, such as

• AlexNet -> VGG -> GoogleNet -> ResNet -> ResNeXt -> DenseNet -> ... ... for classification
• U-Net -> Residual U-Net -> Dense U-Net -> UNet++ -> ... ... for segmentation

But our self-supervised learning framework is independent of these debates, but instead, tells how to empower these models with a generic image feature.

For ease of adoption, we implement our idea on the most popular/reliable publicly available 3D models online (see the acknowledgment section in README). There are some minor differences between 3D models in Keras and PyTorch, but we don't think it would influence much to convey our self-supervised learning framework.

I hope it is helpful.

Zongwei