gnu-octave / statistics

The Statistics package for GNU Octave
GNU General Public License v3.0
24 stars 22 forks source link

Failing test on GitHub runner #113

Closed mmuetzel closed 7 months ago

mmuetzel commented 1 year ago

One of the tests of the statistics package is currently failing on the GitHub runner that checks if the Windows nightly builds are working correctly: https://github.com/gnu-octave/octave-buildbot/actions/runs/5708727265/job/15466671014#step:14:13432

  >>>>> processing D:\a\octave-buildbot\octave-buildbot\octave-2023-07-30-08-59-w64\mingw64\share\octave\packages\statistics-1.6.0\dist_fit\normfit.m
  ***** test
   load lightbulb
   idx = find (lightbulb(:,2) == 0);
   censoring = lightbulb(idx,3) == 1;
   [muHat, sigmaHat] = normfit (lightbulb(idx,1), [], censoring);
   assert (muHat, 9496.595867378575, 1e-12);
   assert (sigmaHat, 3064.021012796456, 1e-12);
  !!!!! test failed
  ASSERT errors for:  assert (muHat,9496.595867378575,1e-12)

    Location  |  Observed  |  Expected  |  Reason
       ()       9496.5959    9496.5959     Abs err 1.819e-12 exceeds tol 1e-12 by 8e-13

This is the following test afaict: https://github.com/gnu-octave/statistics/blob/main/inst/dist_fit/normfit.m#L428-L434

%!test
%! load lightbulb
%! idx = find (lightbulb(:,2) == 0);
%! censoring = lightbulb(idx,3) == 1;
%! [muHat, sigmaHat] = normfit (lightbulb(idx,1), [], censoring);
%! assert (muHat, 9496.59586737857, 1e-11);
%! assert (sigmaHat, 3064.021012796456, 2e-12);

I see that the hard-coded tolerance has been increased by a factor 10 in the meantime.

Would it make sense to change the tolerance in the first assertion from a hard-coded 1e-11 to eps (muHat) or -- to be on the safe side -- to 2*eps (muHat) instead? And similarly for the second assertion, would it make sense to use 5*eps (sigmaHat)?

No big deal. But it might make it a bit clearer that the expected tolerances are indeed in the order of the machine precision for those numbers.