Open oconnor663 opened 3 years ago
My pypy_ci
branch is getting started on this. It does run tests (though it's broken on Windows). Importantly, there seem to be some weird memory errors coming up:
https://github.com/oconnor663/blake3-py/runs/3115660174?check_suite_focus=true
Could be a PyO3 bug here? Some sort of PyPy quirkiness? I'll need to dig into this.
We are seeing errors in Github actions auto test with blake3 py 0.2.0
I'm able to reproduce this error locally in a pypy3 virtualenv. I'll start here:
❯ pytest -vk strided
===================================== test session starts =====================================
platform linux -- Python 3.7.10[pypy-7.3.5-final], pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /tmp/venv.e5V/bin/pypy3
cachedir: .pytest_cache
rootdir: /home/jacko/blake3-py
collected 16 items / 15 deselected / 1 selected
tests/test_blake3.py::test_strided_array_fails FAILED [100%]
========================================== FAILURES ===========================================
__________________________________ test_strided_array_fails ___________________________________
def test_strided_array_fails():
unstrided = numpy.array([1, 2, 3, 4], numpy.uint8)
strided = numpy.lib.stride_tricks.as_strided(unstrided,
shape=[2],
strides=[2])
> assert bytes(strided) == bytes([1, 3])
E AssertionError: assert b'\x01\x02' == b'\x01\x03'
E At index 1 diff: b'\x02' != b'\x03'
E Full diff:
E - b'\x01\x03'
E ? ^
E + b'\x01\x02'
E ? ^
tests/test_blake3.py:141: AssertionError
=================================== short test summary info ===================================
FAILED tests/test_blake3.py::test_strided_array_fails - AssertionError: assert b'\x01\x02' =...
============================== 1 failed, 15 deselected in 0.46s ===============================
I've filed https://github.com/numpy/numpy/issues/19536 related to the issue above. It doesn't appear to be blake3-specific. Not yet clear whether this explains other errors.
Now I doubt the Numpy issue is related. If I add this extremely simple test, it sometimes passes (all 100k iterations) and sometimes fails (after a variable number of iterations, usually ~200):
def test_simple():
expected = "04e0bb39f30b1a3feb89f536c93be15055482df748674b00d26e5a75777702e9"
for i in range(100_000):
print(i)
assert blake3(b"foo").hexdigest() == expected
The failures look like this:
malloc_consolidate(): invalid chunk size
Fatal Python error: Aborted
Stack (most recent call first, approximate line numbers):
File "/home/jacko/blake3-py/tests/test_blake3.py", line 27 in test_simple
File "/tmp/venv.7pC/site-packages/_pytest/python.py", line 176 in pytest_pyfunc_call
File "/tmp/venv.7pC/site-packages/pluggy/callers.py", line 157 in _multicall
File "/tmp/venv.7pC/site-packages/pluggy/manager.py", line 84 in <lambda>
File "/tmp/venv.7pC/site-packages/pluggy/manager.py", line 90 in _hookexec
File "/tmp/venv.7pC/site-packages/pluggy/hooks.py", line 272 in __call__
...
Something very fishy is going on with memory here.
I've narrowed this down to what I believe is a different bug in PyO3: https://github.com/PyO3/pyo3/issues/1736
~After https://github.com/PyO3/maturin/pull/596 ships.~ That fix has shipped.