Questions about external validation on camus

[FredySu]

Dear authors,

I noticed that you did external test on CAMUS, I wonder if it is possible to get more detail about it. I tried to use the official weight of echonet LVEF prediction, I directly crop the size to (112,112). I also did temporal mirroring mentioned in your paper, but the output MSE is really large, instead of 6.82 reported in your paper. I want to know what else did you do on the dataset? like finetune or split dataset or something like calc (who's) mean&std.

thanks.

[ackbar03]

Hi,

Can you check if there is any normalization done on the input? If I recall, there were some differences in the input format of CAMUS and EchoNet dataset. EchoNet was 3 channel but CAMUS was one, and well as some other differences. You can also check if there are any additional details in the paper "Adaptive contrast for image regression in computer-aided disease assessment" which also used CAMUS for external validation (although without temporal mirroring).

[FredySu]

Thank you for your reply. I add normalization to all datas now.

Currently, the MSE of echonet in overall dataset is 8.30, which is 6.82 in your paper, and the R2 is 0.164. While if temporal mirroring is not used, the R2 will drop to 0.046 which is quite different with 40.10% reported in "Adaptive contrast for image regression in computer-aided disease assessment".

The main differences lies in two aspects. The first is that the camus dataet public now consists of 500 patients instead of 450 labeled+50 unlabeled. The second is that paper "Adaptive contrast for image regression in computer-aided disease assessment" says they did data preprocessing pipeline, i have no idea whatelse need to do except for normalization.

If you recall sth about this (e.g. subsample to the same frame size), please let me know. I would like to try, and let results more closer to what was reported in paper.

Thank you very much.

[ackbar03]

Hi,

Can you check that you are only using the apical-4-chamber frames instead of the 2 chamber frames? CAMUS contains both, but EchoNet-Dynamic was trained on 4 chamber only. Also make sure you are using the same normalization according to the EchoNet dynamic paper (e.g. max-min normalization or mean std). The data preprocessing pipeline refers to the original preprocessing used in EchoNet dynamic.

The other possible change is that you can change the frame sampling to 1 instead of 2 and make sure to use the --full_test option instead of --quick_test. However I think it is less likely this is the reason.

The test was conducted on all 500 patients, there were no additional splits for external validation.