search

lindawangg / COVID-Net

COVID-Net Open Source Initiative
Other
1.15k stars 480 forks source link

formula to calculate NPV #83

Closed ankitdata closed 4 years ago

ankitdata commented 4 years ago

Hi, To calculate PPV below is the formula used in eval.py https://github.com/lindawangg/COVID-Net/blob/fd532e7792e6bc474d0de1a615ba82b87d1cae4d/eval.py#L35

what will the formula to calculate NPV and specificity

Description

I tried npvs = [matrix[i,i]/np.sum(matrix[i,:]) if np.sum(matrix[:,i]) else 0 for i in range(len(matrix))] is this corrrect ?

@lindawangg

Thanks

lindawangg commented 4 years ago

negative predictive value is TN / (TN + FN) and specificity is TN / N. so if you wanted to get the npv for covid, it would be (matrix[0, 0:2] + matrix[1, 0:2]) / (matrix[0, 0:2] + matrix[1, 0:2] + matrix[2, 0:2]). TN is the number of normal and pneumonia that weren't classified as covid and FN is the number of covid that was classified as either normal or pneumonia. N would just be the total number of normal and pneumonia samples, which in this case is 200.

ankitdata commented 4 years ago

Hi @lindawangg ,

I tried out formula to calculate NPV for covid output =[0.99029126 0.98958333] is it accurate for Large model? what this two values are representing .

confusion matrix = [[97. 3. 0.] [ 5. 92. 3.] [ 1. 1. 29.]]

specificity = TN / N = [0.51 0.475] | here , TN = (matrix[0, 0:2] + matrix[1, 0:2]) , N =200

I am looking for the values of negative predictive value and specificity like ppv Normal: 0.942, Pneumonia 0.958, COVID-19: 0.906

ankitdata commented 4 years ago

I was able to calculate all as per below for large model:

Sensitivity of each class can be calculated from its TP/(TP+FN) and specificity of each class can be calculated from its TN/(TN+FP) For more information about concept and equations http://en.wikipedia.org/wiki/Sensitivity_and_specificity For multi-class classifiacation, you may use one against all approach.  Suppose there are three classes: C1, C2, and C3 "TP of C1" is all C1 instances that are classified as C1. "TN of C1" is all non-C1 instances that are not classified as C1. "FP of C1" is all non-C1 instances that are classified as C1. "FN of C1" is all C1 instances that are not classified as C1. To find these four terms of C2 or C3 you can replace C1 with C2 or C3.

TP = 30 TN = 197 FP = 3 FN = 1

Sensitivity= TP / TP+FN Specificity= TN / TN+FP PPV= TP / TP+FP NPV = TN / TN + FN

Sensitivity = 30/30+1 = 96.8 Specificity = 197/197+3 = 98.5 PPV= 30/30+3 = 90.9 NPV=197/197+1=99.5

thanks.