McNemar's test can allow us to tell if any close differences in accuracy are actually statistically significant. It's kind of similar to a chi-squared test and a paired t-test.
Suppose we want to check for a difference in accuracy between our Model A and our Model B. We essentially fill the contigency table with counts of test data calls correct by both model A and B, correct by model A but not B, correct by model B but not A, and incorrect by both models.
McNemar's test can allow us to tell if any close differences in accuracy are actually statistically significant. It's kind of similar to a chi-squared test and a paired t-test.
https://en.wikipedia.org/wiki/McNemar%27s_test
Suppose we want to check for a difference in accuracy between our Model A and our Model B. We essentially fill the contigency table with counts of test data calls correct by both model A and B, correct by model A but not B, correct by model B but not A, and incorrect by both models.