search

Murali-group / supervised-grns

Supervised inference of gene regulatory networks from single-cell RNA-sequencing data
GNU General Public License v3.0
0 stars 0 forks source link

Compute precision-recall curves and ROC curves after combining folds #1

Closed tmmurali closed 3 years ago

tmmurali commented 3 years ago

Right now, we compute AUPRC and AUROC for each fold. We should combine the cross-validation scores for all folds into one list, and compute precison-recall and ROC curves and various scores (AUPRC, AUROC, average precision, etc.) on the combined list. This function should be equally applicable to all types of evaluations.

tmmurali commented 3 years ago

@malabikasen please make this change in a branch and create a pull request that @adyprat or I will review.

tmmurali commented 3 years ago

@adyprat to compute AUPRC in BEELINE, we use the PRROC package in R because of the bug we read about in scikit-learn's computation of the AUPRC. In this project, have you used scikit-learn?

tmmurali commented 3 years ago

Replying to my own comment, it does not look like we compute AUPRC at all but only average precision, so the bug does not affect us. Is that correct, @adyprat?

adyprat commented 3 years ago

You're right. I only compute early precision and average precision. Closing this issue since it is fixed in 8dab7e5