search

X-DataInitiative / tick

Module for statistical learning, with a particular emphasis on time-dependent modelling
https://x-datainitiative.github.io/tick/
BSD 3-Clause "New" or "Revised" License
480 stars 105 forks source link

Poisson regression tests fail #495

Open claudio-ICL opened 1 year ago

claudio-ICL commented 1 year ago

Hi @PhilipDeegan

When I run python setup.py build_ext --inplace pytest, I get 5 failures and 1 error: please see below.

Could you please explain why this is happening? Can I simply ignore those failures if I want to open a PR? I only intend to contribute to tick.hawkes and not to tick.linear_model. 

======================================================================
ERROR: tick.linear_model.tests.logistic_regression_test (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: tick.linear_model.tests.logistic_regression_test
Traceback (most recent call last):
  File "/home/claudio/anaconda3/envs/mytick/lib/python3.6/unittest/loader.py", line 428, in _find_test_path
    module = self._get_module_from_name(name)
  File "/home/claudio/anaconda3/envs/mytick/lib/python3.6/unittest/loader.py", line 369, in _get_module_from_name
    __import__(name)
  File "/home/claudio/projects/mytick/tick/linear_model/tests/logistic_regression_test.py", line 7, in <module>
    from sklearn.metrics.ranking import roc_auc_score
ModuleNotFoundError: No module named 'sklearn.metrics.ranking'

======================================================================
FAIL: test_predict (tick.linear_model.tests.linear_regression_test.Test)
...Test LinearRegression predict
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/claudio/projects/mytick/tick/linear_model/tests/linear_regression_test.py", line 370, in test_predict
    learner.predict(X_test), y_pred, decimal=4)
  File "/home/claudio/anaconda3/envs/mytick/lib/python3.6/site-packages/numpy/testing/_private/utils.py", line 1044, in assert_array_almost_equal
    precision=decimal)
  File "/home/claudio/anaconda3/envs/mytick/lib/python3.6/site-packages/numpy/testing/_private/utils.py", line 840, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 4 decimals

Mismatched elements: 5 / 5 (100%)
Max absolute difference: 7.92
Max relative difference: 20.54
 x: array([-1.641 , -0.7238, -1.4996, -5.2949, -3.3918])
 y: array([ 0.084 , -1.4276, -3.1555,  2.6218,  0.3736])

======================================================================
FAIL: test_score (tick.linear_model.tests.linear_regression_test.Test)
...Test LinearRegression predict
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/claudio/projects/mytick/tick/linear_model/tests/linear_regression_test.py", line 382, in test_score
    learner.score(X_test, y_test), 0.793774, places=4)
AssertionError: 0.7932202816957551 != 0.793774 within 4 places

======================================================================
FAIL: test_decision_function (tick.linear_model.tests.poisson_regression_test.Test)
...Test PoissonRegression decision function
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/claudio/projects/mytick/tick/linear_model/tests/poisson_regression_test.py", line 353, in test_decision_function
    learner.decision_function(X_test), y_pred, decimal=4)
  File "/home/claudio/anaconda3/envs/mytick/lib/python3.6/site-packages/numpy/testing/_private/utils.py", line 1044, in assert_array_almost_equal
    precision=decimal)
  File "/home/claudio/anaconda3/envs/mytick/lib/python3.6/site-packages/numpy/testing/_private/utils.py", line 840, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 4 decimals

Mismatched elements: 5 / 5 (100%)
Max absolute difference: 7.21
Max relative difference: 27.47
 x: array([ 2.7299, 10.226 ,  7.4709,  1.4228,  6.4019])
 y: array([1.1448, 5.2194, 0.2624, 4.5525, 6.4168])

======================================================================
FAIL: test_loglik (tick.linear_model.tests.poisson_regression_test.Test)
...Test PoissonRegression loglik function
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/claudio/projects/mytick/tick/linear_model/tests/poisson_regression_test.py", line 364, in test_loglik
    learner.loglik(X_test, y_test), 1.8254, decimal=4)
  File "/home/claudio/anaconda3/envs/mytick/lib/python3.6/site-packages/numpy/testing/_private/utils.py", line 1044, in assert_array_almost_equal
    precision=decimal)
  File "/home/claudio/anaconda3/envs/mytick/lib/python3.6/site-packages/numpy/testing/_private/utils.py", line 840, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 4 decimals

Mismatched elements: 1 / 1 (100%)
Max absolute difference: 0.19
Max relative difference: 0.1
 x: array(2.0107)
 y: array(1.8254)

======================================================================
FAIL: test_predict (tick.linear_model.tests.poisson_regression_test.Test)
...Test PoissonRegression predict
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/claudio/projects/mytick/tick/linear_model/tests/poisson_regression_test.py", line 341, in test_predict
    np.testing.assert_array_almost_equal(learner.predict(X_test), y_pred)
  File "/home/claudio/anaconda3/envs/mytick/lib/python3.6/site-packages/numpy/testing/_private/utils.py", line 1044, in assert_array_almost_equal
    precision=decimal)
  File "/home/claudio/anaconda3/envs/mytick/lib/python3.6/site-packages/numpy/testing/_private/utils.py", line 840, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 6 decimals

Mismatched elements: 4 / 5 (80%)
Max absolute difference: 7.
Max relative difference: 2.
 x: array([ 3., 10.,  7.,  1.,  6.])
 y: array([1., 5., 0., 5., 6.])

----------------------------------------------------------------------
PhilipDeegan commented 1 year ago

the first error I think I tried to fix in an existing PR

you could try making this change in a PR if you like.

the other failures look a bit worry some and I can't really say why they're happening

claudio-ICL commented 1 year ago

Do the second, third... sixth failures happen on your machine too? Or is it a problem local to mine?

PhilipDeegan commented 1 year ago

testing on Linux with python3.9 I do not see any of these test failures besides the roc_auc_score issue, which I will fix shortly in a small PR

PhilipDeegan commented 1 year ago

I've made a PR with some small updates and there are some other random tests failing

this seems pretty consistent across operating systems, I would guess there was some change in 3rd party code which we did not account for

see: https://github.com/X-DataInitiative/tick/actions/runs/3604079340/jobs/6073036140