search

numpy / numpy

The fundamental package for scientific computing with Python.
https://numpy.org
Other
27.85k stars 10.01k forks source link

BUG: In SWIG tests, several OverflowErrors arise from testMaxWrongDim #27577

Open musicinmybrain opened 1 week ago

musicinmybrain commented 1 week ago

Describe the issue:

Working on Fedora 40, with swig 4.2.1, several errors of the form ERROR: testMaxWrongDim (__main__.ucharTestCase.testMaxWrongDim) occur when attempting to run the SWIG tests.

Reproduce the code example:

$ gh repo clone numpy/numpy
$ cd numpy
$ cd tools/swig
$ python3.12 -m venv _e
$ . _e/bin/activate
(_e) $ pip install setuptools numpy
(_e) $ make test

Error message:

======================================================================
ERROR: testMaxWrongDim (__main__.ucharTestCase.testMaxWrongDim)
Test max function with wrong dimensions
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ben/src/forks/numpy/tools/swig/test/testTensor.py", line 103, in testMaxWrongDim
    self.assertRaises(TypeError, max, [0, -1, 2, -3])
  File "/usr/lib64/python3.12/unittest/case.py", line 778, in assertRaises
    return context.handle('assertRaises', args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/unittest/case.py", line 238, in handle
    callable_obj(*args, **kwargs)
  File "/home/ben/src/forks/numpy/tools/swig/test/Tensor.py", line 87, in ucharMax
    return _Tensor.ucharMax(tensor)
           ^^^^^^^^^^^^^^^^^^^^^^^^
OverflowError: Python integer -1 out of bounds for uint8

======================================================================
ERROR: testMaxWrongDim (__main__.ushortTestCase.testMaxWrongDim)
Test max function with wrong dimensions
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ben/src/forks/numpy/tools/swig/test/testTensor.py", line 103, in testMaxWrongDim
    self.assertRaises(TypeError, max, [0, -1, 2, -3])
  File "/usr/lib64/python3.12/unittest/case.py", line 778, in assertRaises
    return context.handle('assertRaises', args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/unittest/case.py", line 238, in handle
    callable_obj(*args, **kwargs)
  File "/home/ben/src/forks/numpy/tools/swig/test/Tensor.py", line 129, in ushortMax
    return _Tensor.ushortMax(tensor)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
OverflowError: Python integer -1 out of bounds for uint16

======================================================================
ERROR: testMaxWrongDim (__main__.uintTestCase.testMaxWrongDim)
Test max function with wrong dimensions
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ben/src/forks/numpy/tools/swig/test/testTensor.py", line 103, in testMaxWrongDim
    self.assertRaises(TypeError, max, [0, -1, 2, -3])
  File "/usr/lib64/python3.12/unittest/case.py", line 778, in assertRaises
    return context.handle('assertRaises', args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/unittest/case.py", line 238, in handle
    callable_obj(*args, **kwargs)
  File "/home/ben/src/forks/numpy/tools/swig/test/Tensor.py", line 171, in uintMax
    return _Tensor.uintMax(tensor)
           ^^^^^^^^^^^^^^^^^^^^^^^
OverflowError: Python integer -1 out of bounds for uint32

======================================================================
ERROR: testMaxWrongDim (__main__.ulongTestCase.testMaxWrongDim)
Test max function with wrong dimensions
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ben/src/forks/numpy/tools/swig/test/testTensor.py", line 103, in testMaxWrongDim
    self.assertRaises(TypeError, max, [0, -1, 2, -3])
  File "/usr/lib64/python3.12/unittest/case.py", line 778, in assertRaises
    return context.handle('assertRaises', args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/unittest/case.py", line 238, in handle
    callable_obj(*args, **kwargs)
  File "/home/ben/src/forks/numpy/tools/swig/test/Tensor.py", line 213, in ulongMax
    return _Tensor.ulongMax(tensor)
           ^^^^^^^^^^^^^^^^^^^^^^^^
OverflowError: Python integer -1 out of bounds for uint64

======================================================================
ERROR: testMaxWrongDim (__main__.ulongLongTestCase.testMaxWrongDim)
Test max function with wrong dimensions
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ben/src/forks/numpy/tools/swig/test/testTensor.py", line 103, in testMaxWrongDim
    self.assertRaises(TypeError, max, [0, -1, 2, -3])
  File "/usr/lib64/python3.12/unittest/case.py", line 778, in assertRaises
    return context.handle('assertRaises', args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/unittest/case.py", line 238, in handle
    callable_obj(*args, **kwargs)
  File "/home/ben/src/forks/numpy/tools/swig/test/Tensor.py", line 255, in ulongLongMax
    return _Tensor.ulongLongMax(tensor)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OverflowError: Python integer -1 out of bounds for uint64

----------------------------------------------------------------------
Ran 324 tests in 0.050s

FAILED (errors=5)

Python and NumPy Versions:

2.1.2
3.12.6 (main, Sep  9 2024, 00:00:00) [GCC 14.2.1 20240801 (Red Hat 14.2.1-1)]

Runtime Environment:

[{'numpy_version': '2.1.2',
  'python': '3.12.6 (main, Sep  9 2024, 00:00:00) [GCC 14.2.1 20240801 (Red '
            'Hat 14.2.1-1)]',
  'uname': uname_result(system='Linux', node='musicbox.musicinmybrain.net', release='6.10.12-200.fc40.x86_64', version='#1 SMP PREEMPT_DYNAMIC Mon Sep 30 21:38:25 UTC 2024', machine='x86_64')},
 {'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
                      'found': ['SSSE3',
                                'SSE41',
                                'POPCNT',
                                'SSE42',
                                'AVX',
                                'F16C',
                                'FMA3',
                                'AVX2'],
                      'not_found': ['AVX512F',
                                    'AVX512CD',
                                    'AVX512_KNL',
                                    'AVX512_KNM',
                                    'AVX512_SKX',
                                    'AVX512_CLX',
                                    'AVX512_CNL',
                                    'AVX512_ICL']}},
 {'architecture': 'Haswell',
  'filepath': '/home/ben/src/forks/numpy/tools/swig/_e/lib/python3.12/site-packages/numpy.libs/libscipy_openblas64_-ff651d7f.so',
  'internal_api': 'openblas',
  'num_threads': 16,
  'prefix': 'libscipy_openblas',
  'threading_layer': 'pthreads',
  'user_api': 'blas',
  'version': '0.3.27'}]

Context for the issue:

I noticed these errors while attempting to investigate a more serious incompatibility with SWIG 4.3. It’s not immediately obvious to me what the correct fix would be.

ngoldbaum commented 2 days ago

Does https://github.com/numpy/numpy/pull/27579 fix this?

musicinmybrain commented 2 days ago

Does #27579 fix this?

Well – it was supposed to, and I could have sworn I validated it, but I can still reproduce this.

seberg commented 1 day ago

@musicinmybrain this is a simple test failure. The test uses self.assertRaises(TypeError, max, [0, -1, 2, -3]), but whatever the underlying function does, it converts the -1 to a uint8, etc.

Just replacing it with 0, 1, 2, 3 (removing the negatives) should be OK, I think... Or use some other random set of positive numbers.

seberg commented 1 day ago

The maybe bigger thing is: apparently we have tests for this swig code (I didn't even know!) and they are not run by our CI...