wagtail / Willow

A wrapper that combines the functionality of multiple Python image libraries into one API
https://willow.wagtail.org/
BSD 3-Clause "New" or "Revised" License
273 stars 53 forks source link

test_opencv.TestOpenCVOperations.test_detect_faces is flaky #134

Closed stephanlachnit closed 8 months ago

stephanlachnit commented 10 months ago

I had a case where the test failed one time with the following error:

FAIL: test_detect_faces (tests.test_opencv.TestOpenCVOperations.test_detect_faces)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/autopkgtest.BPuIbH/autopkgtest_tmp/tests/test_opencv.py", line 74, in test_detect_faces
    assert_allclose(faces, self.expected_faces, atol=2)
  File "/usr/lib/python3/dist-packages/numpy/testing/_private/utils.py", line 1592, in assert_allclose
    assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
  File "/usr/lib/python3.11/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/numpy/testing/_private/utils.py", line 862, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=1e-07, atol=2

Mismatched elements: 8 / 8 (100%)
Max absolute difference: 182
Max relative difference: 1.98901099
 x: array([[ 90, 164, 188, 262],
       [272,  88, 365, 181]], dtype=int32)
 y: array([[272,  89, 364, 181],
       [ 91, 165, 187, 261]])

As can be seen, this is essentially the same array just ordered differently. I think in the context of detecting faces that should be okay? I don't know if there is a why to test this though, maybe one can sort the outer array.

Most of the time, the test does not seem to fail.