HELP WANTED: No covariance estimated using elm (too small std?)

[EveCharbie]

When I try to do

t = spm1d.stats.ttest2(a, b, equal_var=False)

I get the following error message:

  File "/home/charbie/miniconda3/envs/spm/lib/python3.12/site-packages/spm1d/stats/t.py", line 237, in ttest2
    return glm(Y, X, c, Q, roi=roi)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/charbie/miniconda3/envs/spm/lib/python3.12/site-packages/spm1d/stats/t.py", line 62, in glm
    df = _reml.estimate_df_T(Y, X, eij, Q)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/charbie/miniconda3/envs/spm/lib/python3.12/site-packages/spm1d/stats/_reml.py", line 57, in estimate_df_T
    V,h         = reml(YY, X, Q)
    ^^^
TypeError: cannot unpack non-iterable NoneType object

because my data never meet the condition dF < 0.1 l.230 from spm1d/stats/_reml.py, meaning that the function relm returns None. I have a sample with regions of small dispersion (1e-3). The data is attached. Could someone help me fix this issue? data.zip

[0todd0000]

Thank you for reporting this bug! I have not previously seen a dataset for which reml did not converge, so thank you very much for finding and reporting this problem!

Please find that I have updated the code with a fix that returns the result (rather than None) if the total number of iterations K is reached, where K has a default value of 128.

If you used pip to install spm1d then you can upgrade to the most recent version using:

pip install spm1d --upgrade

After upgrading please confirm that you are using the most recent version (0.4.28)

Additional notes:

• The reml procedure does not converge in this case likely due to the data distribution; reml is a parametric procedure that presumes normality
• These data are highly non-normal, with a trimodal distribution or even with more modes
• When the data are non-normal, it is usually best to use a nonparametric procedure (spm1d.stats.nonparam.ttest2 in this case)
• If you change the keyword argument equal_var to True then it will execute without errors
• For this dataset the t-values are so large (almost 60!) so there is effectively no consequnce to using (1) equal_var=True, (2) equal_var=False or (3) spm1d.stats.nonparam.ttest2

[EveCharbie]

Hi :) Thank you very much for the super quick and detailed answer! The fix works as a charm.