Markin-Wang / XProNet

[ECCV2022] The official implementation of Cross-modal Prototype Driven Network for Radiology Report Generation
Apache License 2.0
68 stars 9 forks source link

??pseudo labels?? #20

Open zzduoYI opened 3 months ago

zzduoYI commented 3 months ago

Hello, I am deeply inspired by your work. I would like to ask about the use of pseudo labels, it is understandable to use on the training set, but I do not understand the use on the verification set and the test set. In the real world, we don't have a test set that can generate pseudo labels with CheXbert, so we can't input another feature to the model like an image. Looking forward to your reply, I would appreciate it.

Markin-Wang commented 3 months ago

Hello, I am deeply inspired by your work. I would like to ask about the use of pseudo labels, it is understandable to use on the training set, but I do not understand the use on the verification set and the test set. In the real world, we don't have a test set that can generate pseudo labels with CheXbert, so we can't input another feature to the model like an image. Looking forward to your reply, I would appreciate it.

Hi, Thank you for your interest to our work. XPRONet aims to be applied in a not untypical scenario where the radiologists themselves would be able to straightforwardly provide high-level (structured) disease label information (normally more accurate than the pseudo-labels). This would be both efficient and easier, and our generator produces the full report based on the disease labels and the radiograph. Considering the difficulty in obtaining the ground truth disease labels in the real word, we also veified the performance of XPRONet by utilizing the pseudo-labels from a visual-based classifier trained by the pseudo-labels from the training set. The results can be found in the ReadMe file and make almost no difference to the results using the pseudo-labels from the ChexBert. Hope this would help you figure out the problem.