search

wolph / python-utils

Python Utils is a module with some convenient utilities not included with the standard Python install
http://python-utils.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
91 stars 38 forks source link

Unit test failures on s390x #41

Closed maribu closed 2 weeks ago

maribu commented 4 weeks ago

While updating the package of python3-utils for Alpine to 3.9.0 I stumbled upon unit test failures on s390x. See https://gitlab.alpinelinux.org/maribu/aports/-/jobs/1567033 for the full build log. See https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/73788 for the merge request.

Please note: The unit tests passed on all other platforms supported by Alpine (which all are little endian) and only fails on s390x (which is big endian).

============================= test session starts ==============================
platform linux -- Python 3.12.7, pytest-8.3.3, pluggy-1.5.0
rootdir: /builds/maribu/aports/testing/py3-utils/src/python-utils-3.9.0
configfile: pytest.ini
plugins: asyncio-0.24.0, cov-5.0.0
asyncio: mode=Mode.STRICT, default_loop_scope=None
collected 66 items
_python_utils_tests/test_aio.py ...                                      [  4%]
_python_utils_tests/test_containers.py .....                             [ 12%]
_python_utils_tests/test_decorators.py ....                              [ 18%]
_python_utils_tests/test_generators.py .F..                              [ 24%]
_python_utils_tests/test_import.py ......                                [ 33%]
_python_utils_tests/test_logger.py .                                     [ 34%]
_python_utils_tests/test_python_utils.py .                               [ 36%]
_python_utils_tests/test_time.py ....FFFFF.....                          [ 57%]
python_utils/containers.py ......                                        [ 66%]
python_utils/converters.py ......                                        [ 75%]
python_utils/decorators.py ...                                           [ 80%]
python_utils/exceptions.py .                                             [ 81%]
python_utils/formatters.py ...                                           [ 86%]
python_utils/logger.py ...                                               [ 90%]
python_utils/loguru.py .                                                 [ 92%]
python_utils/terminal.py .                                               [ 93%]
python_utils/time.py ....                                                [100%]
=================================== FAILURES ===================================
_____________________________ test_abatcher_timed ______________________________
    @pytest.mark.asyncio
    async def test_abatcher_timed() -> None:
        batches: types.List[types.List[int]] = []
        async for batch in python_utils.abatcher(
            python_utils.acount(stop=10, delay=0.08), interval=0.1
        ):
            batches.append(batch)

>       assert batches == [[0, 1, 2], [3, 4], [5, 6], [7, 8], [9]]
E       assert [[0, 1], [2, ..., [7], [8, 9]] == [[0, 1, 2], [..., [7, 8], [9]]
E         
E         At index 0 diff: [0, 1] != [0, 1, 2]
E         Left contains one more item: [8, 9]
E         
E         Full diff:
E           [
E               [
E                   0,
E                   1,
E         -         2,
E               ],
E               [
E         +         2,
E                   3,
E         +     ],
E         +     [
E                   4,
E               ],
E               [
E                   5,
E                   6,
E               ],
E               [
E                   7,
E         -         8,
E               ],
E               [
E         +         8,
E                   9,
E               ],
E           ]
_python_utils_tests/test_generators.py:26: AssertionError
______________ test_timeout_generator[0.01-0.006-0.5-0.01-abc-c] _______________
timeout = 0.01, interval = 0.006, interval_multiplier = 0.5
maximum_interval = 0.01, iterable = 'abc', result = 'c'
    @pytest.mark.parametrize(
        'timeout,interval,interval_multiplier,maximum_interval,iterable,result',
        [
            (0.01, 0.006, 0.5, 0.01, 'abc', 'c'),
            (0.01, 0.007, 0.5, 0.01, itertools.count, 2),
            (0.01, 0.007, 0.5, 0.01, itertools.count(), 2),
            (0.01, 0.006, 1.0, None, 'abc', 'c'),
            (
                timedelta(seconds=0.01),
                timedelta(seconds=0.006),
                2.0,
                timedelta(seconds=0.01),
                itertools.count,
                2,
            ),
        ],
    )
    def test_timeout_generator(
        timeout: float,
        interval: float,
        interval_multiplier: float,
        maximum_interval: float,
        iterable: types.Union[
            str,
            types.Iterable[types.Any],
            types.Callable[..., types.Iterable[types.Any]],
        ],
        result: int,
    ) -> None:
        i = None
        for i in python_utils.timeout_generator(
            timeout=timeout,
            interval=interval,
            interval_multiplier=interval_multiplier,
            iterable=iterable,
            maximum_interval=maximum_interval,
        ):
            assert i is not None

>       assert i == result
E       AssertionError: assert 'b' == 'c'
E         
E         - c
E         + b
_python_utils_tests/test_time.py:84: AssertionError
_____________ test_timeout_generator[0.01-0.007-0.5-0.01-count-2] ______________
timeout = 0.01, interval = 0.007, interval_multiplier = 0.5
maximum_interval = 0.01, iterable = <class 'itertools.count'>, result = 2
    @pytest.mark.parametrize(
        'timeout,interval,interval_multiplier,maximum_interval,iterable,result',
        [
            (0.01, 0.006, 0.5, 0.01, 'abc', 'c'),
            (0.01, 0.007, 0.5, 0.01, itertools.count, 2),
            (0.01, 0.007, 0.5, 0.01, itertools.count(), 2),
            (0.01, 0.006, 1.0, None, 'abc', 'c'),
            (
                timedelta(seconds=0.01),
                timedelta(seconds=0.006),
                2.0,
                timedelta(seconds=0.01),
                itertools.count,
                2,
            ),
        ],
    )
    def test_timeout_generator(
        timeout: float,
        interval: float,
        interval_multiplier: float,
        maximum_interval: float,
        iterable: types.Union[
            str,
            types.Iterable[types.Any],
            types.Callable[..., types.Iterable[types.Any]],
        ],
        result: int,
    ) -> None:
        i = None
        for i in python_utils.timeout_generator(
            timeout=timeout,
            interval=interval,
            interval_multiplier=interval_multiplier,
            iterable=iterable,
            maximum_interval=maximum_interval,
        ):
            assert i is not None

>       assert i == result
E       assert 1 == 2
_python_utils_tests/test_time.py:84: AssertionError
___________ test_timeout_generator[0.01-0.007-0.5-0.01-iterable2-2] ____________
timeout = 0.01, interval = 0.007, interval_multiplier = 0.5
maximum_interval = 0.01, iterable = count(2), result = 2
    @pytest.mark.parametrize(
        'timeout,interval,interval_multiplier,maximum_interval,iterable,result',
        [
            (0.01, 0.006, 0.5, 0.01, 'abc', 'c'),
            (0.01, 0.007, 0.5, 0.01, itertools.count, 2),
            (0.01, 0.007, 0.5, 0.01, itertools.count(), 2),
            (0.01, 0.006, 1.0, None, 'abc', 'c'),
            (
                timedelta(seconds=0.01),
                timedelta(seconds=0.006),
                2.0,
                timedelta(seconds=0.01),
                itertools.count,
                2,
            ),
        ],
    )
    def test_timeout_generator(
        timeout: float,
        interval: float,
        interval_multiplier: float,
        maximum_interval: float,
        iterable: types.Union[
            str,
            types.Iterable[types.Any],
            types.Callable[..., types.Iterable[types.Any]],
        ],
        result: int,
    ) -> None:
        i = None
        for i in python_utils.timeout_generator(
            timeout=timeout,
            interval=interval,
            interval_multiplier=interval_multiplier,
            iterable=iterable,
            maximum_interval=maximum_interval,
        ):
            assert i is not None

>       assert i == result
E       assert 1 == 2
_python_utils_tests/test_time.py:84: AssertionError
______________ test_timeout_generator[0.01-0.006-1.0-None-abc-c] _______________
timeout = 0.01, interval = 0.006, interval_multiplier = 1.0
maximum_interval = None, iterable = 'abc', result = 'c'
    @pytest.mark.parametrize(
        'timeout,interval,interval_multiplier,maximum_interval,iterable,result',
        [
            (0.01, 0.006, 0.5, 0.01, 'abc', 'c'),
            (0.01, 0.007, 0.5, 0.01, itertools.count, 2),
            (0.01, 0.007, 0.5, 0.01, itertools.count(), 2),
            (0.01, 0.006, 1.0, None, 'abc', 'c'),
            (
                timedelta(seconds=0.01),
                timedelta(seconds=0.006),
                2.0,
                timedelta(seconds=0.01),
                itertools.count,
                2,
            ),
        ],
    )
    def test_timeout_generator(
        timeout: float,
        interval: float,
        interval_multiplier: float,
        maximum_interval: float,
        iterable: types.Union[
            str,
            types.Iterable[types.Any],
            types.Callable[..., types.Iterable[types.Any]],
        ],
        result: int,
    ) -> None:
        i = None
        for i in python_utils.timeout_generator(
            timeout=timeout,
            interval=interval,
            interval_multiplier=interval_multiplier,
            iterable=iterable,
            maximum_interval=maximum_interval,
        ):
            assert i is not None

>       assert i == result
E       AssertionError: assert 'b' == 'c'
E         
E         - c
E         + b
_python_utils_tests/test_time.py:84: AssertionError
___ test_timeout_generator[timeout4-interval4-2.0-maximum_interval4-count-2] ___
timeout = datetime.timedelta(microseconds=10000)
interval = datetime.timedelta(microseconds=6000), interval_multiplier = 2.0
maximum_interval = datetime.timedelta(microseconds=10000)
iterable = <class 'itertools.count'>, result = 2
    @pytest.mark.parametrize(
        'timeout,interval,interval_multiplier,maximum_interval,iterable,result',
        [
            (0.01, 0.006, 0.5, 0.01, 'abc', 'c'),
            (0.01, 0.007, 0.5, 0.01, itertools.count, 2),
            (0.01, 0.007, 0.5, 0.01, itertools.count(), 2),
            (0.01, 0.006, 1.0, None, 'abc', 'c'),
            (
                timedelta(seconds=0.01),
                timedelta(seconds=0.006),
                2.0,
                timedelta(seconds=0.01),
                itertools.count,
                2,
            ),
        ],
    )
    def test_timeout_generator(
        timeout: float,
        interval: float,
        interval_multiplier: float,
        maximum_interval: float,
        iterable: types.Union[
            str,
            types.Iterable[types.Any],
            types.Callable[..., types.Iterable[types.Any]],
        ],
        result: int,
    ) -> None:
        i = None
        for i in python_utils.timeout_generator(
            timeout=timeout,
            interval=interval,
            interval_multiplier=interval_multiplier,
            iterable=iterable,
            maximum_interval=maximum_interval,
        ):
            assert i is not None

>       assert i == result
E       assert 1 == 2
_python_utils_tests/test_time.py:84: AssertionError
---------- coverage: platform linux, python 3.12.7-final-0 -----------
Name                         Stmts   Miss Branch BrPart  Cover   Missing
------------------------------------------------------------------------
python_utils/__about__.py        6      0      0      0   100%
python_utils/__init__.py        13      0      0      0   100%
python_utils/aio.py             30      0     12      0   100%
python_utils/containers.py     123      0     62      0   100%
python_utils/converters.py      81      0     40      0   100%
python_utils/decorators.py      44      0     16      0   100%
python_utils/exceptions.py       7      0      0      0   100%
python_utils/formatters.py      36      0     24      0   100%
python_utils/generators.py      39      0     18      0   100%
python_utils/import_.py         34      0     16      0   100%
python_utils/logger.py          48      0     48      0   100%
python_utils/loguru.py          10      0      0      0   100%
python_utils/terminal.py         9      0      0      0   100%
python_utils/time.py            98      0     42      0   100%
python_utils/types.py           15      0      0      0   100%
------------------------------------------------------------------------
TOTAL                          593      0    278      0   100%
Required test coverage of 100.0% reached. Total coverage: 100.00%
=========================== short test summary info ============================
FAILED _python_utils_tests/test_generators.py::test_abatcher_timed - assert [[0, 1], [2, ..., [7], [8, 9]] == [[0, 1, 2], [..., [7, 8], [9]]

  At index 0 diff: [0, 1] != [0, 1, 2]
  Left contains one more item: [8, 9]

  Full diff:
    [
        [
            0,
            1,
  -         2,
        ],
        [
  +         2,
            3,
  +     ],
  +     [
            4,
        ],
        [
            5,
            6,
        ],
        [
            7,
  -         8,
        ],
        [
  +         8,
            9,
        ],
    ]
FAILED _python_utils_tests/test_time.py::test_timeout_generator[0.01-0.006-0.5-0.01-abc-c] - AssertionError: assert 'b' == 'c'

  - c
  + b
FAILED _python_utils_tests/test_time.py::test_timeout_generator[0.01-0.007-0.5-0.01-count-2] - assert 1 == 2
FAILED _python_utils_tests/test_time.py::test_timeout_generator[0.01-0.007-0.5-0.01-iterable2-2] - assert 1 == 2
FAILED _python_utils_tests/test_time.py::test_timeout_generator[0.01-0.006-1.0-None-abc-c] - AssertionError: assert 'b' == 'c'

  - c
  + b
FAILED _python_utils_tests/test_time.py::test_timeout_generator[timeout4-interval4-2.0-maximum_interval4-count-2] - assert 1 == 2
========================= 6 failed, 60 passed in 6.39s =========================
wolph commented 3 weeks ago

Similar to your other issue I think timings are the culprit here. I don't think endianness is what caused these issues, all of the failing tests I see here are very time-sensitive currently.

I've tried to keep the test-suite as fast as possible, but that does mean that a different platform (possibly emulated in your case?) can have timing issues. I'm going to try and fiddle with the numbers a bit so we're less likely to fail.

wolph commented 3 weeks ago

I'm curious if the timing relaxations in the latests tests have fixed these issues as well, can you test them please? :)

maribu commented 2 weeks ago

Yes, indeed. So this was not endian related, but just the same as https://github.com/wolph/python-utils/issues/42