mahmoodlab / MMP

Multimodal prototyping for cancer survival prediction - ICML 2024
Other
43 stars 7 forks source link

About the results #1

Closed H-Q-N closed 1 week ago

H-Q-N commented 1 month ago

Thank you for your contribution. Regarding the c-index value of the experiment, it does not reach the result in your paper.Use resnet 50, x20, 256×256patches in CLAM for feature extraction. Multimodal Fusion : model_tuple='PANTHER,default'、model_mm_type='coattn'. Data Set:TCGA-BRCA Here are my results:

epoch 50 : 0.649±0.046
epoch 100 : 0.655±0.046
epoch 200 :0.651±0.050

The c_index_test result of the five-fold cross validation corresponding to epoch100 is:

k=0  0.572164948453608
k=1  0.645803698435277
k=2  0.663324538258575
k=3  0.691629955947137
k=4  0.703731911652704

Hope to get your reply.

andrewsong90 commented 1 month ago

Hi @H-Q-N ,

Could you refer to which table you are referring to? I don't think I report BRCA results for ResNet50 with pathways.

H-Q-N commented 1 month ago

你好@H-Q-N,

您能参考一下您指的是哪个表格吗?我认为我没有使用路径报告 ResNet50 的 BRCA 结果。

Snipaste_2024-07-21_21-26-37 This table. The experimental results obtained are very different from this

andrewsong90 commented 1 month ago

Yes the results in Table 1 are based on UNI feature encoder https://github.com/mahmoodlab/UNI

With Resnet50, the results are going to be lower than UNI (One of our findings in the study - Table 8).

Hope this information helps

H-Q-N commented 1 month ago

Yes the results in Table 1 are based on UNI feature encoder https://github.com/mahmoodlab/UNI

With Resnet50, the results are going to be lower than UNI (One of our findings in the study - Table 8).

Hope this information helps

Thank you very much for your reply. I have some questions about other places. One question is, is the result in Table 1 the maximum value or the average value of the five-fold cross validation? I found an interesting situation that as the k value increases, the c-index value also increases. Another question is: for the division of the five-fold cross validation, is it random or customized?

andrewsong90 commented 1 month ago

These are average value across the five folds (hence the mean and the standard deviation).

I think that c-index value increasing with k value is purely out of coincidence. I am pretty sure for other cancer types, this won't be the case.

The current splits are based on site-stratification predominantly (Since TCGA has many different data collection sites) - We did not want the same tissue site to appear both in train and test split, as this can be considered as data leakage (https://www.nature.com/articles/s41467-021-24698-1)