How to get AUC

[XuanhaoYang]

Hello, I have read your article "PreMSlm: An R package for predicting microsatellite instability from the expression profiling of a gene panel in cancer". It is a nice tool for MSI prediction using RNA-seq data! I am wondering how you evaluate the AUC score of your tool in different database. The output of PreMSlm is only "0" or "1". Did you have the probability of the corresponding result of "0"/"1"? Yours, Xuanhao Yang

[WangX-Lab]

Hello,

Thank you for your interest in our tool.

Yes, we calculated the probability of the corresponding result of "0/1" first, and according to the definition of AUC, the AUC results of each dataset were calculated with the probability of 0.5 as the cutoff. We did not put the probability value into the final prediction result of the software, because our focus is on the binary classification output of the samples. If you want to obtain the AUC values of your own datasets, you can refer to the source code of the function msi_pre(): knn.pred <- knn(training_data[,-1], input_data, training_data[,1], k = 5, prob = TRUE), and use the code: attributes(knn.pred)$prob, to calculate.

With the best regards,

Xiaosheng Wang, MD, PhD

China Pharmaceutical University

发件人: @. @.> 代表 XuanhaoYang 发送时间: Wednesday, November 17, 2021 9:59 PM 收件人: WangX-Lab/PreMSIm @.> 抄送: Subscribed @.> 主题: [WangX-Lab/PreMSIm] How to get AUC (Issue #4)

Hello, I have read your article "PreMSlm: An R package for predicting microsatellite instability from the expression profiling of a gene panel in cancer". It is a nice tool for MSI prediction using RNA-seq data! I am wondering how you evaluate the AUC score of your tool in different database. The output of PreMSlm is only "0" or "1". Did you have the probability of the corresponding result of "0"/"1"? Yours, Xuanhao Yang

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/WangX-Lab/PreMSIm/issues/4 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDDKVHWPUDXLWMDXUINLZ3UMOYLJANCNFSM5IHBOOCQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

[XuanhaoYang]

Thanks for your quick rely! Can you just futher explain how you caculate the AUC score using the the code: attributes(knn.pred)$prob? Because I use the attributes(knn.pred)$prob yesterday in the same database as your article pointed but didn't get the same AUC as yours. What code did you use to caculate the AUC? Very thank you, looking forward to your rely.

[WangX-Lab]

Hello,

Please let me know what dataset you used, and I will give you the detailed code for the corresponding dataset.

With the best regards,

Xiaosheng Wang, MD, PhD

China Pharmaceutical University

发件人: @. @.> 代表 XuanhaoYang 发送时间: Thursday, November 18, 2021 11:29 AM 收件人: WangX-Lab/PreMSIm @.> 抄送: WangX-Lab @.>; Comment @.***> 主题: Re: [WangX-Lab/PreMSIm] How to get AUC (Issue #4)

Thanks for your quick rely! Can you just futher explain how you caculate the AUC score using the the code: attributes(knn.pred)$prob? Because I use the attributes(knn.pred)$prob yesterday in the same database as your article pointed but didn't get the same AUC as yours. What code did you use to caculate the AUC? Very thank you, looking forward to your rely.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/WangX-Lab/PreMSIm/issues/4#issuecomment-972486163 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDDKVDZSMNIYX73KKLKQXDUMRXG7ANCNFSM5IHBOOCQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

[XuanhaoYang]

Hello, maybe Colorectal cancer (GSE13067) dataset as an example. I used PreMSIm to GSE13067 dataset and got this result by attributes(knn.pred)$prob. As you can see from the chart, I don't see clear relationship with the "MSI_status" and "MSI_pro" result.

[WangX-Lab]

Hello,

I see your confusion. In fact, the probability calculated by the code attributes(knn.pred)$prob, refers to the probability of the binary classification result —— MSI status. For example (according to the data frame you gave), if MSI_status = 1, the value of MSI_prob means how likely is it that the status is "1". And if MSI_status = 0, the value of MSI_prob means how likely is it that the status is "0", meanwhile the value of 1 - MSI_prob means how likely is it that the status is "1".

With the best regards,

Xiaosheng Wang, MD, PhD

China Pharmaceutical University

发件人: @. @.> 代表 XuanhaoYang 发送时间: Friday, November 19, 2021 10:08 AM 收件人: WangX-Lab/PreMSIm @.> 抄送: WangX-Lab @.>; Comment @.***> 主题: Re: [WangX-Lab/PreMSIm] How to get AUC (Issue #4)

Hello, maybe Colorectal cancer (GSE13067) dataset as an example. https://user-images.githubusercontent.com/68040500/142552318-c0e0d878-7c4a-4d57-8f1c-88b0b2c77d82.png I used PreMSIm to GSE13067 dataset and got this result by attributes(knn.pred)$prob. As you can see from the chart, I don't see clear relationship with the "MSI_status" and "MSI_pro" result.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/WangX-Lab/PreMSIm/issues/4#issuecomment-973660957 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDDKVFT2S53MHXZY42L6ITUMWWPFANCNFSM5IHBOOCQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .