search

Markin-Wang / XProNet

[ECCV2022] The official implementation of Cross-modal Prototype Driven Network for Radiology Report Generation
Apache License 2.0
65 stars 9 forks source link

how to train the model to get the result same as the paper？ #3

Closed wyh196646 closed 1 year ago

wyh196646 commented 2 years ago

Dear Author，I'm really enthusiastic about your fantastic work，I just train IU-Xray dataset use your code，and I get the best result is test_BLEU_1 : 0.432083288799186 test_BLEU_2 : 0.27494955972033686 test_BLEU_3 : 0.19862342676532663 test_BLEU_4 : 0.1530441598906721 test_METEOR : 0.17428250108359328 test_ROUGE_L : 0.3539938538730797 and in training process ，the terminal show ‘Validation performance didn't improve for 50 epochs. Training stops.‘ In the same time, I juest test the model with the trained models you provide, I get the result same as the paper,I‘m really confused ,So,could you supply more details about how to train the best model? Sincerely

Markin-Wang commented 2 years ago

Dear Author，I'm really enthusiastic about your fantastic work，I just train IU-Xray dataset use your code，and I get the best result is test_BLEU_1 : 0.432083288799186 test_BLEU_2 : 0.27494955972033686 test_BLEU_3 : 0.19862342676532663 test_BLEU_4 : 0.1530441598906721 test_METEOR : 0.17428250108359328 test_ROUGE_L : 0.3539938538730797 and in training process ，the terminal show ‘Validation performance didn't improve for 50 epochs. Training stops.‘ In the same time, I juest test the model with the trained models you provide, I get the result same as the paper,I‘m really confused ,So,could you supply more details about how to train the best model? Sincerely

Hi, thanks for your interest. Please train the model via the provided script: run_iu_xray.sh. I have tried my best to ensure the reproducibility of our work, e.g., fixed seed and use the determinstic algorithm. Hence, the results are exactly the same for each run in my workstation. I would recommend you to set the same environment of ours. In addition, as pointed by other repos, e.g.,R2Gen, the language model is highly sensitive to the hyper-parameters, and the IU X-Ray dataset is small, so the variance of the results may be larger. You can try to tune some hyper-parameters.

wyh196646 commented 2 years ago

Really appreciate for your clear reponse！

njan-creative commented 1 year ago

R2GenResults

I am also getting results similar to the values in this bug.

LangWY commented 1 year ago

Hello, I also have the same question. I wonder if you have achieved the results on the IU-Xray dataset now.If successful, I hope you can tell me how you solved it, for example, which hyperparameter have been adjusted.

Markin-Wang commented 1 year ago

Hello, I also have the same question. I wonder if you have achieved the results on the IU-Xray dataset now.If successful, I hope you can tell me how you solved it, for example, which hyperparameter have been adjusted.

Hi, thanks for your interest. I would recommend you to tune the learning rate first. Note that the language model is very sensitive to the learning rate, especially on the small dataset.

LangWY commented 1 year ago

Thank you for your answer. Your model is really effective, and I will try according to your suggestions.

baoning1208 commented 1 year ago

谢谢你的回答。你的模型真的很有效，我会按照你的建议尝试。

This is indeed a great job, dear comrade, if you get good results, please let me know your hyperparameters