Closed harpergith closed 1 year ago
Hi, thank you very much for following our work! I checked the training details of SIIM segmentation and adjusted some details of the learning rate and schedule, especially for finetuning on 10% data. I hope the adjustments are helpful, referring to https://github.com/RL4M/MRM-pytorch/tree/main/Siim_Segmentation/configs/mrm. Thank you again and welcome to ask new questions to help us improve the code.
Thank you for your quick response. I tried to use the new configuration file "upernet_mrm-base_fp16_8x2_512x512_160k_siim_10per.py" and "schedule_5k" to finetune on 10% data, but got even lower results (67.03%) than before (69.1%).
I'm sorry for missing an important file when extracting core files from the complex code structure. See new commit(https://github.com/RL4M/MRM-pytorch/commit/694e67925f40e59af514819d439319142d2a1870) and (https://github.com/RL4M/MRM-pytorch/commit/96f7fc31749c1368a28b04b6a87cdefd2968067d) to get correct loss functions.
@harpergith
Hi Harpergith,
Have you tried our new commit? Are there any problems with your experiments? If you can reproduce our results, we would close this issue now.
Best,
Hi, Thank you for your quick efforts in improving the code. Now I can reproduce the results when finetuning on 10% data.
Hi, as observed in the SIIM segmentation task, the finetuning configurations matter a lot on the performance. Do you have plan to release the fine-tuning code/configurations of other classification datasets, such as RSNA pneumonia?
Hi, the code and other hyperparameters are the same for other classification datasets. It may be helpful to see https://github.com/RL4M/MRM-pytorch/blob/3730ba49a678eda870284205178bfe25627aaed8/README.md?plain=1#L87 for the references of warm-up steps, total steps, and learning rate.
Thanks for your advice.
Best regards,
Thank you for the update. This is very helpful.
Best regards,
Hi, thank you for your efforts in organizing and releasing the code. I tried the code on the SIIM segmentation task by carefully following the instructions on organizing/splitting the dataset and setting up the open mmsegmentation framework. However, using the provided configuration files, the results cannot be reproduced.
The pre-trained weights MRM.pth are downloaded from the provided link. Finetuning on 100% SIIM data gave Dice 90.7% and finetuning on 10% data gave Dice 69.1%, which are lower than the values in the paper (91.4% and 73.2%) especially when finetuning on 10% data.
Could you clarify how to reproduce the results? Thanks.