[BUG] Partially reducing unpacked numpy array causes segmentation fault.

[hylkedonker]

Bug description

I've come across another small code snippet that causes mojo to seg fault. It is a strange combination of several necessary conditions:

1. A fn function returning a tuple.
2. This tuple being composed of numpy matrices.
3. Partial reduction of the returned matrices using the instance's method.

Steps to reproduce

Here is the corresponding code snippet:

from python import Python

fn f() raises -> Tuple[PythonObject, PythonObject]:
    var np = Python.import_module("numpy")
    var a = np.array([1, 2, 3, 4]).reshape(2, 2)
    var b = np.array([3, 4, 5, 6]).reshape(2, 2)
    return a, b

def main():
    ab = f()
    a_avg = ab[0].mean(axis=0)
    print(a_avg) # Segmentation fault.

And hereby the segmentation fault error message:

[91563:91563:20240511,205612.485274:ERROR elf_dynamic_array_reader.h:64] tag not found [91563:91563:20240511,205612.485724:ERROR elf_dynamic_array_reader.h:64] tag not found [91563:91563:20240511,205612.486038:ERROR elf_dynamic_array_reader.h:64] tag not found [91563:91563:20240511,205612.486308:ERROR elf_dynamic_array_reader.h:64] tag not found [91563:91563:20240511,205612.486458:ERROR elf_dynamic_array_reader.h:64] tag not found [91563:91563:20240511,205612.486639:ERROR elf_dynamic_array_reader.h:64] tag not found [91563:91563:20240511,205612.486837:ERROR elf_dynamic_array_reader.h:64] tag not found [91563:91563:20240511,205612.487000:ERROR elf_dynamic_array_reader.h:64] tag not found [91563:91563:20240511,205612.487144:ERROR elf_dynamic_array_reader.h:64] tag not found [91563:91563:20240511,205612.487310:ERROR elf_dynamic_array_reader.h:64] tag not found [91563:91563:20240511,205612.487457:ERROR elf_dynamic_array_reader.h:64] tag not found [91563:91563:20240511,205612.487646:ERROR elf_dynamic_array_reader.h:64] tag not found [91563:91563:20240511,205612.487794:ERROR elf_dynamic_array_reader.h:64] tag not found [91563:91563:20240511,205612.487943:ERROR elf_dynamic_array_reader.h:64] tag not found [91563:91563:20240511,205612.488091:ERROR elf_dynamic_array_reader.h:64] tag not found [91563:91563:20240511,205612.488241:ERROR elf_dynamic_array_reader.h:64] tag not found [91563:91563:20240511,205612.488419:ERROR elf_dynamic_array_reader.h:64] tag not found Please submit a bug report to https://github.com/modularml/mojo/issues and include the crash backtrace along with all the relevant source codes. Stack dump:

0. Program arguments: /home/user/.modular/pkg/packages.modular.com_mojo/bin/mojo /home/user/path/to/myfile.mojo Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var LLVM_SYMBOLIZER_PATH to point to it): 0 mojo 0x00005a1a83a1c407 1 mojo 0x00005a1a83a1a25e 2 mojo 0x00005a1a83a1ca9f 3 libc.so.6 0x0000786360242990 4 libpython3.11.so.1.0 0x000078635c3d1c18 5 libpython3.11.so.1.0 0x000078635c4d5f09 _PyObject_GC_NewVar + 121 6 libpython3.11.so.1.0 0x000078635c3dad68 PyTuple_New + 56 7 libpython3.11.so.1.0 0x000078635c3082a6 _PyEval_EvalFrameDefault + 4198 8 libpython3.11.so.1.0 0x000078635c467452 9 libpython3.11.so.1.0 0x000078635c37b768 _PyObject_Call + 280 10 libpython3.11.so.1.0 0x000078635c30b41e _PyEval_EvalFrameDefault + 16862 11 libpython3.11.so.1.0 0x000078635c467452 12 libpython3.11.so.1.0 0x000078635c37b768 _PyObject_Call + 280 13 libpython3.11.so.1.0 0x000078635c30b41e _PyEval_EvalFrameDefault + 16862 14 libpython3.11.so.1.0 0x000078635c467452 15 libpython3.11.so.1.0 0x000078635c30b41e _PyEval_EvalFrameDefault + 16862 16 libpython3.11.so.1.0 0x000078635c467452 17 libpython3.11.so.1.0 0x000078635c37b768 _PyObject_Call + 280 18 libpython3.11.so.1.0 0x000078635c5a3375 19 libpython3.11.so.1.0 0x000078635c3784f1 _PyObject_MakeTpCall + 129 20 libpython3.11.so.1.0 0x000078635c37aca2 21 libpython3.11.so.1.0 0x000078635c37b0a3 PyObject_CallFunctionObjArgs + 163 22 libpython3.11.so.1.0 0x000078635c3ccb66 PyObject_Str + 102 23 libpython3.11.so.1.0 0x00007863080060b4 PyObject_Str + 18446744072296306100 mojo crashed! Please file a bug report. [1] 91561 segmentation fault (core dumped) '/home/user/.modular/pkg/packages.modular.com_mojo/bin/mojo'

Interestingly, when some of the conditions listed above is not met, the problem disappears.

1. Without a tuple return type the following snippet runs okay:

fn g() raises -> PythonObject:
    var np = Python.import_module("numpy")
    var c = np.array([1, 2, 3, 4]).reshape(2, 2)
    return c  # Numpy array instead of a tuple thereof. 

def main():
    c = g()
    print(c.mean(axis=0))   # Okay!

1. Using numpy vectors instead of matrices works okay.

fn h() raises -> Tuple[PythonObject, PythonObject]:
    var np = Python.import_module("numpy")
    var a = np.array([1, 2, 3, 4]) # Vector instead of matrix.
    var b = np.array([3, 4, 5, 6])
    return a, b

def main():
    ab = h()
    a_avg = ab[0].mean(axis=0)
    print(a_avg) # Okay!

1. Using a function instead of a method also works okay:

fn f() raises -> Tuple[PythonObject, PythonObject]:
    var np = Python.import_module("numpy")
    var a = np.array([1, 2, 3, 4]).reshape(2, 2)
    var b = np.array([3, 4, 5, 6]).reshape(2, 2)
    return a, b

def main():
    var np = Python.import_module("numpy")
    ab = f()
    a_avg = np.mean(ab[0], axis=0)  # Function instead of method.
    print(a_avg)  #  Okay!

Or, when a complete reduction is performed, also no segmentation fault occurs:

fn f() raises -> Tuple[PythonObject, PythonObject]:
    var np = Python.import_module("numpy")
    var a = np.array([1, 2, 3, 4]).reshape(2, 2)
    var b = np.array([3, 4, 5, 6]).reshape(2, 2)
    return a, b

def main():
    ab = f()
    a_avg = ab[0].mean() # Complete reduction.
    print(a_avg) # Okay!

So a bit of a mystery, it seems. :-)

System information

OS: Ubuntu 23.10
`mojo -v`: mojo 24.3.0 (9882e19d)
`modular -v`: modular 0.7.4 (df7a9e8b)

[hylkedonker]

I think the following simpler problem maybe related:

from python import Python, PythonObject

fn h() raises -> Tuple[PythonObject, PythonObject]:
    var np = Python.import_module("numpy")
    return np.array([]), np.array([])

def main():
    w, h = h()

which crashes with the error:

Reference count error detected: an attempt was made to deallocate the dtype 12 (d)

I am using the nightly build:

mojo 2024.8.2805 (fd3bceba)