search

JinmiaoChenLab / Batch-effect-removal-benchmarking

A benchmark of batch-effect correction methods for single-cell RNA sequencing data
70 stars 44 forks source link

Question about choosing metrics to evaluate BBKNN #8

Open HelloWorldLTY opened 3 years ago

HelloWorldLTY commented 3 years ago

Sorry to disturb you again. I notice that in this paper you do not use any metric to evaluate the effect of BBKNN, I guess this is because bbknn cannot really modify the original count matrix. However, it can affect the result after UMAP dimension reduction. Therefore, could I use the LISI rate and kBET rate to evaluate this method? Thanks a lot.

jinmiaochen commented 3 years ago

Hello, thanks a lot for the question. And thanks for pointing this out. We have checked our scripts and realized the reason why BBKNN was not evaluated using any quantitative metric is that at that point of time, we were unable to compute PCA based on BBKNN’s output which is a graph and it did not produce low dimensional latent representation or corrected values. In our evaluation pipeline, we first computed the PCA using outputs from various batch effect removal methods and used the top 20 PC as inputs to calculate the respective kBET, LISI, ASW, and ARI scores. Good news is that the latest version of BBKNN produces latent representations. And yes, you could use kBET and LISI to evaluate this method.

From: ChineseBest @.> Sent: Tuesday, July 6, 2021 8:34 AM To: JinmiaoChenLab/Batch-effect-removal-benchmarking @.> Cc: Subscribed @.***> Subject: [JinmiaoChenLab/Batch-effect-removal-benchmarking] Question about choosing metrics to evaluate BBKNN (#8)

Sorry to disturb you again. I notice that in this paper you do not use any metric to evaluate the effect of BBKNN, I guess this is because bbknn cannot really modify the original count matrix. However, it can affect the result after UMAP dimension reduction. Therefore, could I use the LISI rate and kBET rate to evaluate this method? Thanks a lot.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/JinmiaoChenLab/Batch-effect-removal-benchmarking/issues/8, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACD33WQEYAMGIK2RSWNGXH3TWJFPBANCNFSM473Q6EUQ.

This e-mail and any attachments are only for the use of the intended recipient and may contain material that is confidential, privileged and/or protected by the Official Secrets Act. If you are not the intended recipient, please delete it or notify the sender immediately. Please do not copy or use it for any purpose or disclose the contents to any other person.