search

cython / cython

The most widely used Python to C compiler
https://cython.org
Apache License 2.0
9.55k stars 1.49k forks source link

[BUG] running test suite fails on armv7l #6424

Open mcepl opened 1 month ago

mcepl commented 1 month ago

Describe the bug

While packaging python-h5py for openSUSE, the running of test suite fails completely on armv7l:

[  664s] + export PYTHONPATH=/home/abuild/rpmbuild/BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages
[  664s] + PYTHONPATH=/home/abuild/rpmbuild/BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages
[  664s] + export PYTHONDONTWRITEBYTECODE=1
[  664s] + PYTHONDONTWRITEBYTECODE=1
[  664s] + pytest-3.10 /home/abuild/rpmbuild/BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages/h5py/ -k 'not (TestMPI or test_float_round_tripping or test_register_filter)'
[  665s] ============================= test session starts ==============================
[  665s] platform linux -- Python 3.10.15, pytest-8.3.2, pluggy-1.5.0
[  665s] rootdir: /home/abuild/rpmbuild
[  665s] collected 745 items / 6 deselected / 739 selected
[  665s] 
[  665s] ../../BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages/h5py/tests/test_attribute_create.py . [  0%]
[  665s] ......                                                                   [  0%]
[  665s] ../../BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages/h5py/tests/test_attrs.py . [  1%]
[  666s] .........................                                                [  4%]
[  666s] ../../BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages/h5py/tests/test_attrs_data.py . [  4%]
[  666s] ......................                                                   [  7%]
[  666s] ../../BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages/h5py/tests/test_base.py . [  7%]
[  666s] ..........                                                               [  9%]
[  666s] ../../BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages/h5py/tests/test_big_endian_file.py . [  9%]
[  666s] .                                                                        [  9%]
[  666s] ../../BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages/h5py/tests/test_completions.py . [  9%]
[  666s] .                                                                        [  9%]
[  666s] ../../BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages/h5py/tests/test_dataset.py . [  9%]
[  666s] s....................................................................... [ 19%]
[  666s] .......s................................................................ [ 29%]
[  667s] .............................s............                               [ 34%]
[  667s] ../../BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages/h5py/tests/test_dataset_getitem.py . [ 35%]
[  667s] ........................................................................ [ 44%]
[  667s] ..........................s...                                           [ 48%]
[  667s] ../../BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages/h5py/tests/test_dataset_swmr.py . [ 48%]
[  667s] .........                                                                [ 50%]
[  667s] ../../BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages/h5py/tests/test_datatype.py . [ 50%]
[  667s] .                                                                        [ 50%]
[  667s] ../../BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages/h5py/tests/test_dimension_scales.py . [ 50%]
[  667s] ......................                                                   [ 53%]
[  667s] ../../BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages/h5py/tests/test_dims_dimensionproxy.py . [ 53%]
[  667s]                                                                          [ 53%]
[  667s] ../../BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages/h5py/tests/test_dtype.py . [ 53%]
[  667s] Fatal Python error: Bus error
[  667s] 
[  667s] Current thread 0xf73bd020 (most recent call first):
[  667s]   File "/home/abuild/rpmbuild/BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages/h5py/_hl/dataset.py", line 999 in __setitem__
[  667s]   File "/home/abuild/rpmbuild/BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages/h5py/tests/test_dtype.py", line 59 in test_compound_vlen_bool
[  667s]   File "/usr/lib/python3.10/unittest/case.py", line 549 in _callTestMethod
[  667s]   File "/usr/lib/python3.10/unittest/case.py", line 591 in run
[  667s]   File "/usr/lib/python3.10/unittest/case.py", line 650 in __call__
[  667s]   File "/usr/lib/python3.10/site-packages/_pytest/unittest.py", line 351 in runtest
[  667s]   File "/usr/lib/python3.10/site-packages/_pytest/runner.py", line 174 in pytest_runtest_call
[  667s]   File "/usr/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
[  667s]   File "/usr/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
[  667s]   File "/usr/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
[  667s]   File "/usr/lib/python3.10/site-packages/_pytest/runner.py", line 242 in <lambda>
[  667s]   File "/usr/lib/python3.10/site-packages/_pytest/runner.py", line 341 in from_call
[  667s]   File "/usr/lib/python3.10/site-packages/_pytest/runner.py", line 241 in call_and_report
[  667s]   File "/usr/lib/python3.10/site-packages/_pytest/runner.py", line 132 in runtestprotocol
[  667s]   File "/usr/lib/python3.10/site-packages/_pytest/runner.py", line 113 in pytest_runtest_protocol
[  667s]   File "/usr/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
[  667s]   File "/usr/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
[  667s]   File "/usr/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
[  667s]   File "/usr/lib/python3.10/site-packages/_pytest/main.py", line 362 in pytest_runtestloop
[  667s]   File "/usr/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
[  667s]   File "/usr/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
[  667s]   File "/usr/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
[  667s]   File "/usr/lib/python3.10/site-packages/_pytest/main.py", line 337 in _main
[  667s]   File "/usr/lib/python3.10/site-packages/_pytest/main.py", line 283 in wrap_session
[  667s]   File "/usr/lib/python3.10/site-packages/_pytest/main.py", line 330 in pytest_cmdline_main
[  667s]   File "/usr/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
[  667s]   File "/usr/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
[  667s]   File "/usr/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
[  667s]   File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 175 in main
[  667s]   File "/usr/lib/python3.10/site-packages/_pytest/config/__init__.py", line 201 in console_main
[  667s]   File "/usr/bin/pytest-3.10", line 8 in <module>
[  667s] 
[  667s] Extension modules: h5py._errors, h5py.defs, h5py._objects, h5py.h5, numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, h5py.utils, h5py.h5t, h5py.h5s, h5py.h5ac, h5py.h5p, h5py.h5r, h5py._proxy, h5py._conv, h5py.h5z, h5py.h5a, h5py.h5d, h5py.h5ds, h5py.h5g, h5py.h5i, h5py.h5f, h5py.h5fd, h5py.h5pl, h5py.h5o, h5py.h5l, h5py._selector (total: 37)
[  667s] /var/tmp/rpm-tmp.Sljaz0: line 52:  5434 Bus error               pytest-3.10 /home/abuild/rpmbuild/BUILDROOT/python-h5py-3.11.0-60.28.arm/usr/lib/python3.10/site-packages/h5py/ -k "not (TestMPI or $donttest)"

Complete build log with all packages used and log of all steps taken to reproduce.

Everything works correctly on other platforms.

Code to reproduce the behaviour:

see above, just run the test suite

Expected behaviour

test suite passes on all platforms

OS

Linux (openSUSE/Tumbleweed)

Python version

3.10.15 (but probably obsereveable on others)

Cython version

3.0.11

Additional context

No response

da-woods commented 1 month ago

It's definitely possible that this is a Cython issue (I think we'd like to test some non-x64 architectures but it proved hard to do).

But it definitely isn't obvious that it is from what you posted. It'd think it's just as likely to be in HDF5 or an error in their code that calls it.

I don't want to dismiss it out-of-hand but I don't think we're the people to start investigating this.

mcepl commented 1 month ago

On Mon Oct 14, 2024 at 10:27 PM CEST, da-woods wrote:

It's definitely possible that this is a Cython issue (I think we'd like to test some non-x64 architectures but it proved hard to do).

But it definitely isn't obvious that it is from what you posted. It'd think it's just as likely to be in HDF5 or an error in their code that calls it.

Possible, I just hoped that you would know better how much you care about armv7l and/or where to ask for more questions better than this overloaded Python package maintainer.

da-woods commented 1 month ago

how much you care about armv7l

In principle it should work and we'd want to fix any bugs stopping it from working. However it's poorly tested on our side.

where to ask for more questions better than this overloaded Python package maintainer.

My uninformed guess is that it's an alignment error (i.e. it's trying to use data that isn't aligned like arm wants it to be). The first thing I'd do to debug it would be to build it with C debug symbols and try to catch it in gdb.

My thought is that the h5py maintainers would be best to make the first pass at that - they probably understand their build system and tests