search

Digital-Dermatology / t-loss

Official code for Robust T-Loss for Medical Image Segmentation (MICCAI 2023)
https://robust-tloss.github.io/
Apache License 2.0
40 stars 6 forks source link

The ISIC training dice is stuck at 0.38 #3

Closed xyimaging closed 9 months ago

xyimaging commented 10 months ago

Hi authors, thanks for your good work and congrats on the best paper award!

I'm trying to reimplement the training on ISIC with alpha = 0.7 and beta = 0.7 following your instruction, at the first several epochs the training seems to be ok and dice kept increasing, however, after the epoch 12 the dice fell off a cliff and was stuck at 0.38. Could you please give any insight about this? Thanks! image image image

alvarogonjim commented 10 months ago

Hi @xyimaging, thank you for your interest in our work. To ensure the reproducibility of the results, we recommend using our own implementation.

Regarding your question, a possible explanation is that in your implementation, $\nu$ reaches a plateau, but the model continues to push it to lower values. This suggests that the model might be overfitting to the noisy labels (resulting in a drop of the Dice score to 0.38). To address this issue, you could record the values of $\nu$ at the iteration level, as we do, and adapt the learning rate based on the behavior of $\nu. Please refer to: https://github.com/Digital-Dermatology/t-loss/blob/f132e8767c2191463bbcc2c15310cd8df1b7ab30/train.py#L312-L323

xyimaging commented 10 months ago

Thanks for your quick reply! I'll have a try.