test_TxComp.py:104: AssertionError

[deniskristak]

Describe the bug Getting error test_TxComp.py:104: AssertionError when testing nanocompore-1.0.0rc3-2 installation

To Reproduce

1. Install nanocompore for Python 3.8
2. Go to nanocompore-1.0.0rc3-2/tests
3. run pytest.py -vv

Expected behavior All tests should pass

Desktop (please complete the following information):

• OS: ubuntu 20.4

Additional context I suspect this may be because the accepted result tolerance is lower since Python 3.5+?

error log:

========================================================================================================= FAILURES =========================================================================================================
__________________________________________________________________________________________________ test_txComp_GMM_anova ___________________________________________________________________________________________________

test_ref_pos_list = ([{'data': {'KD': {'KD1': {'coverage': 100, 'dwell': array([121.70757477, 129.70100881, 111.67411923, 131.1979314 ,
  ...328765287e-39, 3.3968653938213694e-40, 1.9321679678623975e-36, 8.482777798353687e-40, nan, 7.06503867181238e-40, ...]})

    def test_txComp_GMM_anova(test_ref_pos_list):
        ml = mock.Mock()
        if sys.version_info < (3, 6):
            tol = 0.001
        else:
            tol=0.00000001
        res = txCompare("ref_id", test_ref_pos_list[0], methods=['GMM'], logit=False, sequence_context=2, min_coverage=3, logger=ml, allow_warnings=False, random_state=np.random.RandomState(seed=42))
        GMM_pvalues = [pos['txComp']['GMM_anova_pvalue'] for pos in res ]
>       assert GMM_pvalues == [pytest.approx(i, abs=tol, nan_ok=True) for i in test_ref_pos_list[1]['GMM_anova']]
E       assert [0.0008574768473501677,\n 0.0036329291397528157,\n 0.007312047981252302,\n 0.0017335646468135102,\n nan,\n 0.004197519576768562,\n 0.004730678586860965,\n 0.0028228474915020945,\n 0.0023262178697710987,\n 0.00020764199465021126] == [0.0008574768473501677  1.0e-08,\n 0.0036329291397528157  1.0e-08,\n 0.007312047981252302  1.0e-08,\n 0.0010906844025473576  1.0e-08,\n nan  1.0e-08,\n 0.004197519576768562  1.0e-08,\n 0.004730678586860965  1.0e-08,\n 0.0028228474915020945  1.0e-08,\n 0.0023262178697710987  1.0e-08,\n 0.00020764199465021126  1.0e-08]
E         At index 3 diff: 0.0017335646468135102 != 0.0010906844025473576  1.0e-08
E         Full diff:
E           [
E         -  0.0008574768473501677  1.0e-08,
E         ?                       ----------
E         +  0.0008574768473501677,
E         -  0.0036329291397528157  1.0e-08,
E         ?                       ----------
E         +  0.0036329291397528157,
E         -  0.007312047981252302  1.0e-08,
E         ?                      ----------
E         +  0.007312047981252302,
E         -  0.0010906844025473576  1.0e-08,
E         -  nan  1.0e-08,
E         +  0.0017335646468135102,
E         +  nan,
E         -  0.004197519576768562  1.0e-08,
E         ?                      ----------
E         +  0.004197519576768562,
E         -  0.004730678586860965  1.0e-08,
E         ?                      ----------
E         +  0.004730678586860965,
E         -  0.0028228474915020945  1.0e-08,
E         ?                       ----------
E         +  0.0028228474915020945,
E         -  0.0023262178697710987  1.0e-08,
E         ?                       ----------
E         +  0.0023262178697710987,
E         -  0.00020764199465021126  1.0e-08,
E         ?                        ----------
E         +  0.00020764199465021126,
E           ]

test_TxComp.py:104: AssertionError

[tleonardi]

Hi @deniskristak, thanks for the detailed bug report! The difference between expected and actual results seems a little too big to be a tolerance problem.. I'll look into it as soon as possible. Thanks for spotting it!

[deniskristak]

Hi, @tleonardi thanks a lot, if it will help you with debugging, this is an easybuild PR I used it in - you can see what other dependencies I'm using https://github.com/easybuilders/easybuild-easyconfigs/pull/11690

[boegel]

@deniskristak Maybe try installing nanocompore with the GCC-based foss/2020a toolchain too, and see if you also hit the failing test there?

This could be a case of too aggressive optimization by the Intel compilers...

[deniskristak]

@boegel @tleonardi results with foss/2020a are the same

[tleonardi]

Hi @deniskristak, the problem should now be fixed in devel. I'll make a new release with the fix soon.