search

lloongx / SFDA-CBMT

Pytorch implementation of MICCAI'23 pape: Source-Free Domain Adaptive Fundus Image Segmentation with Class-Balanced Mean Teacher.
14 stars 0 forks source link

Question about the uploaded model #7

Open BarY7 opened 1 week ago

BarY7 commented 1 week ago

Hi @lloongx , thank you for the paper! I have a question regarding the uploaded trained model from Domain3. It seems like the W/o DA performance of this model is too high compared to the results of the paper (especially for Domain1, which closely achieves the results after then method has been applied)

These are my results for Domain1, Domain2 after downloading the dataset and model:

Domain1:

../datasets/Fundus/Domain1/train/ROIs/image
Number of images in train/ROIs: 50
../datasets/Fundus/Domain1/train/ROIs/image
Number of images in train/ROIs: 50
../datasets/Fundus/Domain1/test/ROIs/image
Number of images in test/ROIs: 51
==> Loading DeepLab model file: ./logs_train/Domain3/source_model.pth.tar
initial dice: cup: 0.8455+-0.1265 disc: 0.9644+-0.0168 avg: 0.9049, assd: cup: 10.1335+-6.6095 disc: 4.0838+-1.9258 avg: 7.108

Domain2:

../datasets/Fundus/Domain2/train/ROIs/image
Number of images in train/ROIs: 99
../datasets/Fundus/Domain2/train/ROIs/image
Number of images in train/ROIs: 99
../datasets/Fundus/Domain2/test/ROIs/image
Number of images in test/ROIs: 60
==> Loading DeepLab model file: ./logs_train/Domain3/source_model.pth.tar
initial dice: cup: 0.7677+-0.1621 disc: 0.8878+-0.0378 avg: 0.8278, assd: cup: 9.6556+-4.7018 disc: 10.4599+-3.7296 avg: 10.0578
lloongx commented 1 week ago

We follow the same training strategy from SFDA-DPL on the source domain. However, due to the slight differences between our model and the model from SFDA-DPL (you can refer to the model definition), we can't reuse its source model. Therefore, we reproduce its source domain training procedure. For consistent comparisons, we use the "W/o DA" score from the SFDA-DPL paper.

On Domain1, the initial scores are already very close to the upper bound, so the improvements are marginal.

BarY7 commented 6 days ago

Thanks for answering @lloongx !

Could you elaborate regarding the source model change (why was it needed?)

I did not see a mention of this in the experiments section of the paper. In the code I see that the decoder part is indeed different in the last conv layer - is that the change you mentioned or is there anything else?

lloongx commented 3 days ago

Sorry for late reply.

the decoder part is indeed different in the last conv layer

You are right, it's where I change the model. The implementation in SFDA-DPL is from BEAL and is different from original deeplabv3+, so we restore it to the right version.