Open DopamineLcy opened 1 year ago
Hi, thanks for your question.
Yes, data leakage might occur. One could try fine-tune on MIMIC-IV and see the effect of data leakage. But the pretaining task (predict token masking) is very different from icd coding task (multi label classification), which is why pretrained Clinical-Longformer (w/o HSAP &Prompt row in table 1) does not outperform unpretrained but task-specific engineered model such as JointLAAT.
p.s. We also explored pretraining on other private corpus with phrase and sentence masking and then fientuned on MIMIC-III. So no leakage in this paper. Results are shown here.
Hi, thank you for your nice work! But I have question that the model is initialized with Clinical-Longformer (https://huggingface.co/yikuan8/Clinical-Longformer), which is trained with mimic-iii note data. Is there any risk of data leak, i.e. the test set is appeared during the pre-training of Clinical-Longformer? Thank you very much.
Best,