rasbt / mlxtend

A library of extension and helper modules for Python's data analysis and machine learning libraries.
https://rasbt.github.io/mlxtend/
Other
4.82k stars 853 forks source link

mcnemar(tb, exact=True) do not return None as chi2 #989

Closed ftnext closed 1 year ago

ftnext commented 1 year ago

Thanks for your wonderful work! I noticed a bug.

Describe the bug

mlxtend.evaluate.mcnemar does not return None as chi2 when exact=True.

Steps/Code to Reproduce

ref: https://rasbt.github.io/mlxtend/user_guide/evaluate/mcnemar/#example-3-mcnemars-test-for-scenario-a

>>> import numpy as np
>>> from mlxtend.evaluate import mcnemar

>>> tb_a = np.array([[9959, 11], [1, 29]])
>>> tb_a
array([[9959,   11],
       [   1,   29]])
>>> chi2, _ = mcnemar(tb_a, exact=True)

Expected Results

chi2 is None.

ref: https://rasbt.github.io/mlxtend/user_guide/evaluate/mcnemar/#api

if exact=True (default: False), chi2 is None

Actual Results

chi2 is not None.

>>> chi2
1

The value comes from:

https://github.com/rasbt/mlxtend/blob/1943512dbf68ed967ec5a8941d6ce8d96288ff09/mlxtend/evaluate/mcnemar.py#L216

Unit test does not verify that the return value is None.

https://github.com/rasbt/mlxtend/blob/1943512dbf68ed967ec5a8941d6ce8d96288ff09/mlxtend/evaluate/tests/test_mcnemar_test.py#L44

The assertion should be assert chi2p is None

Versions

MLxtend 0.21.0 macOS-10.14.6-x86_64-i386-64bit Python 3.10.2 (v3.10.2:a58ebcc701, Jan 13 2022, 14:50:16) [Clang 13.0.0 (clang-1300.0.29.30)] Scikit-learn 1.1.2 NumPy 1.23.3 SciPy 1.9.1