math-a3k / covid-ht

covid-ht aims to provide a tool for implementing an AI layer on clinical classification effectively in order to improve detection, information availability and resource efficiency in medical environments.
https://covid-ht.herokuapp.com/
GNU Lesser General Public License v3.0
2 stars 3 forks source link

Make specificity available as a CV metric #20

Closed math-a3k closed 2 years ago

math-a3k commented 2 years ago

Sensitivity and Specificity are the default standards for evaluating diagnostic tests (https://en.wikipedia.org/wiki/Sensitivity_and_specificity), sensitivity is called "recall" but specificity is not provided in https://scikit-learn.org/stable/modules/model_evaluation.html#classification-metrics but it can be obtained through the confusion matrix or the recall of the negative class.

Implement in way that can be incorporated in the cv_metrics field of a classifier (i.e. "accuracy, recall, specificity"):

get_cross_validation_scores() is currently considered to be engine specific:

More info: https://scikit-learn.org/stable/modules/model_evaluation.html#scoring-parameter

math-a3k commented 2 years ago

This hit a limitation / flaw in django-ai (https://github.com/math-a3k/django-ai/commit/fd85385254cd146a6097854739226259935042e5), with that settled, it is implemented in https://github.com/math-a3k/covid-ht/commit/fbfb98387df27bea1b0610b8b4d8f6a628f7eeda.