I coded the Area Under the Precision-Recall Curves (AUPRC) as a measure to evaluate cross-validation. I ran several checks and it is working correctly for me. The only limitation is that we could not account for sampling weights when calculating the AUPRC, however, I included a warning to highlight this limitation. I send below a few references:
Fu, G. H., Yi, L. Z., & Pan, J. (2019). Tuning model parameters in class‐imbalanced learning with precision‐recall curve. Biometrical Journal, 61(3), 652-664.
Fu, G. H., Xu, F., Zhang, B. Y., & Yi, L. Z. (2017). Stable variable selection of class-imbalanced data with precision-recall criterion. Chemometrics and Intelligent Laboratory Systems, 171, 241-250.
Dear Jared,
I coded the Area Under the Precision-Recall Curves (AUPRC) as a measure to evaluate cross-validation. I ran several checks and it is working correctly for me. The only limitation is that we could not account for sampling weights when calculating the AUPRC, however, I included a warning to highlight this limitation. I send below a few references:
Fu, G. H., Yi, L. Z., & Pan, J. (2019). Tuning model parameters in class‐imbalanced learning with precision‐recall curve. Biometrical Journal, 61(3), 652-664.
Fu, G. H., Xu, F., Zhang, B. Y., & Yi, L. Z. (2017). Stable variable selection of class-imbalanced data with precision-recall criterion. Chemometrics and Intelligent Laboratory Systems, 171, 241-250.
Kind regards, Pedro