search

KaveIO / PhiK

Phi_K correlation analyzer library
Other
156 stars 28 forks source link

Failing tests #73

Closed fabaff closed 10 months ago

fabaff commented 11 months ago

On NixOS with Python 3.11 we get some failing tests. I think that the integration tests are failing because of an issue at our end but the ones with the TypeError looks like an issue with numpy.


Executing pytestCheckPhase
============================= test session starts ==============================
platform linux -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /build/source
plugins: anyio-4.0.0
collected 17 items                                                             

tests/phik_python/test_phik.py ............FF.                           [ 88%]
tests/phik_python/integration/test_notebooks.py FF                       [100%]

=================================== FAILURES ===================================
___________________ PhiKTest.test_significance_matrix_hybrid ___________________
joblib.externals.loky.process_executor._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/nix/store/497dy4kviy0gcssvas0h98nyjn57arbp-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/externals/loky/process_executor.py", line 463, in _process_worker
    r = call_item()
        ^^^^^^^^^^^
  File "/nix/store/497dy4kviy0gcssvas0h98nyjn57arbp-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/externals/loky/process_executor.py", line 291, in __call__
    return self.fn(*self.args, **self.kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/497dy4kviy0gcssvas0h98nyjn57arbp-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py", line 589, in __call__
    return [func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/497dy4kviy0gcssvas0h98nyjn57arbp-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py", line 589, in <listcomp>
    return [func(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/fhvkyqhzz2k9jqcfapx6cnnzvfjry3xq-python3.11-phik-0.12.3/lib/python3.11/site-packages/phik/simulation.py", line 170, in _simulate_and_fit
    simdata = sim_data(exp_dep, method=simulation_method)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/fhvkyqhzz2k9jqcfapx6cnnzvfjry3xq-python3.11-phik-0.12.3/lib/python3.11/site-packages/phik/simulation.py", line 131, in sim_data
    return sim_2d_data(data)
           ^^^^^^^^^^^^^^^^^
  File "/nix/store/fhvkyqhzz2k9jqcfapx6cnnzvfjry3xq-python3.11-phik-0.12.3/lib/python3.11/site-packages/phik/simulation.py", line 50, in sim_2d_data
    hout = np.random.multinomial(n=ndata, pvals=hcr)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "numpy/random/mtrand.pyx", line 4256, in numpy.random.mtrand.RandomState.multinomial
TypeError: 'numpy.float64' object cannot be interpreted as an integer
"""

The above exception was the direct cause of the following exception:

self = <phik_python.test_phik.PhiKTest testMethod=test_significance_matrix_hybrid>

    def test_significance_matrix_hybrid(self):
        """Test significance calculation"""

        # open fake car insurance data
        df = pd.read_csv(resources.fixture("fake_insurance_data.csv.gz"))
        cols = list(df.columns)
        # get significances
        interval_cols = ["driver_age", "mileage"]
>       sm = df.significance_matrix(
            interval_cols=interval_cols, significance_method="hybrid"
        )

tests/phik_python/test_phik.py:206: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/nix/store/fhvkyqhzz2k9jqcfapx6cnnzvfjry3xq-python3.11-phik-0.12.3/lib/python3.11/site-packages/phik/significance.py:421: in significance_matrix
    return significance_from_rebinned_df(
/nix/store/fhvkyqhzz2k9jqcfapx6cnnzvfjry3xq-python3.11-phik-0.12.3/lib/python3.11/site-packages/phik/significance.py:352: in significance_from_rebinned_df
    pvalue, zvalue = significance_from_hist2d(
/nix/store/fhvkyqhzz2k9jqcfapx6cnnzvfjry3xq-python3.11-phik-0.12.3/lib/python3.11/site-packages/phik/significance.py:278: in significance_from_hist2d
    pvalue, zvalue = significance_from_chi2_hybrid(
/nix/store/fhvkyqhzz2k9jqcfapx6cnnzvfjry3xq-python3.11-phik-0.12.3/lib/python3.11/site-packages/phik/significance.py:204: in significance_from_chi2_hybrid
    chi2s = sim_chi2_distribution(
/nix/store/fhvkyqhzz2k9jqcfapx6cnnzvfjry3xq-python3.11-phik-0.12.3/lib/python3.11/site-packages/phik/simulation.py:161: in sim_chi2_distribution
    chi2s = Parallel(n_jobs=njobs)(delayed(_simulate_and_fit)(exp_dep, simulation_method, lambda_)
/nix/store/497dy4kviy0gcssvas0h98nyjn57arbp-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py:1952: in __call__
    return output if self.return_generator else list(output)
/nix/store/497dy4kviy0gcssvas0h98nyjn57arbp-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py:1595: in _get_outputs
    yield from self._retrieve()
/nix/store/497dy4kviy0gcssvas0h98nyjn57arbp-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py:1699: in _retrieve
    self._raise_error_fast()
/nix/store/497dy4kviy0gcssvas0h98nyjn57arbp-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py:1734: in _raise_error_fast
    error_job.get_result(self.timeout)
/nix/store/497dy4kviy0gcssvas0h98nyjn57arbp-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py:736: in get_result
    return self._return_or_raise()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <joblib.parallel.BatchCompletionCallBack object at 0x7fff699894d0>

    def _return_or_raise(self):
        try:
            if self.status == TASK_ERROR:
>               raise self._result
E               TypeError: 'numpy.float64' object cannot be interpreted as an integer

/nix/store/497dy4kviy0gcssvas0h98nyjn57arbp-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py:754: TypeError
[...]
___________________ PipelineNotebookTest.test_basic_tutorial ___________________

self = <phik_python.integration.test_notebooks.PipelineNotebookTest testMethod=test_basic_tutorial>

    def test_basic_tutorial(self):
>       self.run_notebook(resources.notebook("phik_tutorial_basic.ipynb"))

tests/phik_python/integration/test_notebooks.py:21: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/phik_python/bases.py:31: in run_notebook
    self.assertTrue(status, "Notebook execution failed (%s)" % notebook)
E   AssertionError: False is not true : Notebook execution failed (/nix/store/fhvkyqhzz2k9jqcfapx6cnnzvfjry3xq-python3.11-phik-0.12.3/lib/python3.11/site-packages/phik/notebooks/phik_tutorial_basic.ipynb)
----------------------------- Captured stderr call -----------------------------
[IPKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled
=============================== warnings summary ===============================
../../nix/store/fhvkyqhzz2k9jqcfapx6cnnzvfjry3xq-python3.11-phik-0.12.3/lib/python3.11/site-packages/phik/resources.py:18
  /nix/store/fhvkyqhzz2k9jqcfapx6cnnzvfjry3xq-python3.11-phik-0.12.3/lib/python3.11/site-packages/phik/resources.py:18: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    from pkg_resources import resource_filename

../../nix/store/fdp87b2p91w3xy11jigwwd431d7hq24y-python3.11-jupyter_client-8.3.1/lib/python3.11/site-packages/jupyter_client/connect.py:20
  /nix/store/fdp87b2p91w3xy11jigwwd431d7hq24y-python3.11-jupyter_client-8.3.1/lib/python3.11/site-packages/jupyter_client/connect.py:20: DeprecationWarning: Jupyter is migrating its paths to use standard platformdirs
  given by the platformdirs library.  To remove this warning and
  see the appropriate new directories, set the environment variable
  `JUPYTER_PLATFORM_DIRS=1` and then run `jupyter --paths`.
  The use of platformdirs will be the default in `jupyter_core` v6
    from jupyter_core.paths import jupyter_data_dir, jupyter_runtime_dir, secure_write

tests/phik_python/test_phik.py: 1420 warnings
  /nix/store/fhvkyqhzz2k9jqcfapx6cnnzvfjry3xq-python3.11-phik-0.12.3/lib/python3.11/site-packages/phik/bivariate.py:85: DeprecationWarning: Please use `mvnun` from the `scipy.stats` namespace, the `scipy.stats.mvn` namespace is deprecated.
    return mvn.mvnun(lower, upper, mu, S)[0]

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/phik_python/test_phik.py::PhiKTest::test_significance_matrix_hybrid - TypeError: 'numpy.float64' object cannot be interpreted as an integer
FAILED tests/phik_python/test_phik.py::PhiKTest::test_significance_matrix_mc - TypeError: 'numpy.float64' object cannot be interpreted as an integer
FAILED tests/phik_python/integration/test_notebooks.py::PipelineNotebookTest::test_advanced_tutorial - AssertionError: False is not true : Notebook execution failed (/nix/store/f...
FAILED tests/phik_python/integration/test_notebooks.py::PipelineNotebookTest::test_basic_tutorial - AssertionError: False is not true : Notebook execution failed (/nix/store/f...
================= 4 failed, 13 passed, 1422 warnings in 24.98s =================
``
mbaak commented 10 months ago

Thanks for reporting, I will have a look and get back to you 👍 .

mbaak commented 10 months ago

Hey @fabaff, looking at this now. Do you happen to have/know a NixOS docker image (that includes Python 3.11) I could use for running the latest phik tests? Thanks!

mbaak commented 10 months ago

Fyi I fixed the TypeError, now in the master branch. It's now passing an integer to np.random.multinomial(). The other one I haven't been able to reproduce yet. I'll ping you when the new patch release is out (later this week).

mbaak commented 10 months ago

Closing this now. If the notebook failure remains, please ping me, with a NixOS docker image I can reproduce this in :-)