Missing TCGA-NSCLC and TCGA-RCC code

[apardyl]

Hi, I was trying to reproduce your results, but I see that the code for TCGA-NSCLC and TCGA-RCC datasets is missing. Could you add missing experiment setups?

[RuixiangZhao]

@apardyl Hi, I'm also trying to reproduce the author's experiment. I have finished the experiment on Camelyon16, but there are some gaps between the my experimental results and the results in the author's paper. Have you encountered this situation？ Do you have any advice?

This is my test results on Camelyon16:

test_Accuracy = 0.6976743936538696 test_CohenKappa = 0.31783050298690796 test_F1 = 0.6536316871643066 test_Recall = 0.6494898200035095 test_Precision = 0.6814241409301758 auc = 0.7441326379776001

class 0: acc 0.85, correct 68/80 class 1: acc 0.4489795918367347, correct 22/49

[CAI-XvKun]

I got a similar testing result with AUC 0.74.

[RuixiangZhao]

@CAI-XvKun At first, I thought it was due to my improper feature extraction method. Then I tried three different parameter configurations in CLAM to extract feature and found that the test results in TransMIL were similar:

So, I think there may be some details that we have not yet noticed.

If you make any progress, please tell me. Thank you very much!

[szc19990412]

Please carefully check whether the resolution of the image is 20 times, and the process of extracting features should follow the procedure of CLAM. And whether the labels of the images are consistent.

[RuixiangZhao]

@szc19990412 Thank you very much for your advice!

I'm curious about x20 magnification. When I tried to extract features from Camelyon16 dataset using CLAM, it seemed that there is no parameters in CLAM that can be directly used to indicate the magnification, instead I were using --patch_level 2. I have tried the following command:

python create_patches_fp.py --source DATA_DIRECTORY --save_dir RESULTS_DIRECTORY --patch_level 2 --patch_size 256 --seg --patch --stitch

And The test results are shown in the first figure above. Do you think this command is correct to extract features of patches at x20 magnification?

And what do you mean "And whether the labels of the images are consistent."? The label file I were using is provided by you fold0.csv.

Thank you very much for your wonderful work!

[szc19990412]

@RuixiangZhao You should use Openslide to look at the resolution information contained in the slide. As far as I know, Camelyon 16 contains a series of resolution information of 40 times, 20 times, etc. And also, the patch level represents the selected resolution, if you want to use the image at x20 magnification, for Camelyon16, you should use --patch_level 1.(Maybe you need to be more familiar with CLAM)

[RuixiangZhao]

@szc19990412 Thankyou for your help, I will learn CLAM and TransMIL further.

[apardyl]

Hi, I believe this discussion went a little bit off topic. My original question was about TCGA-NSCLC and TCGA-RCC datasets 😀

[szc19990412]

@apardyl They are the same as camel dataset, You can process the data and label into the same format as camel

[LXYTSOS]

@apardyl Hi, I'm also trying to reproduce the author's experiment. I have finished the experiment on Camelyon16, but there are some gaps between the my experimental results and the results in the author's paper. Have you encountered this situation？ Do you have any advice?

This is my test results on Camelyon16:

test_Accuracy = 0.6976743936538696 test_CohenKappa = 0.31783050298690796 test_F1 = 0.6536316871643066 test_Recall = 0.6494898200035095 test_Precision = 0.6814241409301758 auc = 0.7441326379776001

class 0: acc 0.85, correct 68/80 class 1: acc 0.4489795918367347, correct 22/49

same here, can't reproduce the experiment results.

[genhao3]

@RuixiangZhao You should use Openslide to look at the resolution information contained in the slide. As far as I know, Camelyon 16 contains a series of resolution information of 40 times, 20 times, etc. And also, the patch level represents the selected resolution, if you want to use the image at x20 magnification, for Camelyon16, you should use --patch_level 1.(Maybe you need to be more familiar with CLAM)

what is CLAM?