search

robbert-harms / MDT

Microstructure Diffusion Toolbox
GNU Lesser General Public License v3.0
48 stars 18 forks source link

Test fails #30

Closed eglerean closed 3 years ago

eglerean commented 4 years ago

hello

I am sorry if my question is very trivial, I am a beginner and trying to test if the tool works as it should. I think the installation went fine, but when running

python tests/test_example_data.py

I get that both tests are failing, here one of them: 

FAIL: test_lls_multishell_b6k_max (__main__.ExampleDataTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test_example_data.py", line 108, in test_lls_multishell_b6k_max
    np.testing.assert_allclose(test_values['mean'], np.mean(roi),
  File "/scratch/work/eglerean/PSC/MDT/env/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 1527, in assert_allclose
    assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
  File "/scratch/work/eglerean/PSC/MDT/env/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 840, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=0.0001, atol=0
b1k_b2k - CHARMED_r1 - LogLikelihood - mean
Mismatched elements: 1 / 1 (100%)
Max absolute difference: 7595.9631958
Max relative difference: 17.17264368
 x: array(-8038.29248)
 y: NiftiInfoDecoratedArray(-442.32928, dtype=float32)

Do you know what could be the reason? How can I debug this?

thank you so much! Enrico

robbert-harms commented 3 years ago

Hi Enrico,

The problem here is that I haven't updated the tests in ages. At the time I hoped I could make the tests work with an Integrated Testing environment, but because that required OpenCL it didn't work. Over time the optimization methods improved a bit which may have brought the test results out of line.

I would recommend ignoring the tests for now and just run MDT on some of your datasets and visually inspect the results or compare with other tools. Does that help you?

Best wishes,

Robbert

eglerean commented 3 years ago

Hi Robert

Thank you so much for your reply. I will proceed with some of our data. Maybe I will do a pull request if I have time to update the tests myself. :)

I close this.

Best Enrico