Issue with concatenating structured arrays (segmentation fault) (Trac #2084)

[numpy-gitbot]

Original ticket http://projects.scipy.org/numpy/ticket/2084 on 2012-03-19 by atmention:astrofrog, assigned to unknown.

The following demonstrates the issue - even though Numpy knows how to concatenate the fields of a structured array, it crashes if one tries to concatenate the structured arrays themselves:

In [1]: import numpy as np

In [2]: d1 = np.array(zip(['a','b','c']), dtype=[('b', '|S1')])

In [3]: d2 = np.array(zip(['aa','bb','cc']), dtype=[('b', '|S2')])

In [4]: np.hstack([d1['b'],d2['b']])
Out[4]: 
array(['a', 'b', 'c', 'aa', 'bb', 'cc'], 
      dtype='|S2')

In [5]: np.hstack([d1, d2])
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/Users/tom/<ipython-input-5-bd5cc420043d> in <module>()
----> 1 np.hstack([d1, d2])

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/shape_base.pyc in hstack(tup)
    268 
    269     """
--> 270     return _nx.concatenate(map(atleast_1d,tup),1)
    271 

TypeError: invalid type promotion

A similar issue occurs with floating-point values of different endian-ness:

In [1]: import numpy as np

In [2]: d1 = np.array(zip([1,2,3]), dtype=[('a', '<f4')])

In [3]: d2 = np.array(zip([1,2,3]), dtype=[('a', '>f4')])

In [4]: np.hstack([d1['a'],d2['a']])
Out[4]: array([ 1.,  2.,  3.,  1.,  2.,  3.], dtype=float32)

In [5]: np.hstack([d1, d2])
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/Users/tom/<ipython-input-5-bd5cc420043d> in <module>()
----> 1 np.hstack([d1, d2])

/Users/tom/Library/Python/2.7/lib/python/site-packages/numpy/core/shape_base.pyc in hstack(tup)
    271     # As a special case, dimension 0 of 1-dimensional arrays is "horizontal"
    272     if arrs[0].ndim == 1:
--> 273         return _nx.concatenate(arrs, 0)
    274     else:
    275         return _nx.concatenate(arrs, 1)

TypeError: invalid type promotion

In some cases, this can even cause segmentation faults, though I have yet to find a way to reproduce this consistently.

[numpy-gitbot]

trac user lcampagn wrote on 2012-09-24

I can confirm this bug and reproduce the segmentation fault:

>>> import numpy as np
>>> a = np.empty(1, dtype=[('x', object)])
>>> b = np.empty(1, dtype=[('x', float)])
>>> np.concatenate([a,b])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: invalid type promotion
>>> np.concatenate([a,b])
Segmentation fault (core dumped)

I've run into this crash in a few different instances. I can cause this crash in both Linux (1.6.1-6ubuntu1) and on Windows. I have not been able to generate useful stack traces--it would appear this bug causes some stack corruption.

[numpy-gitbot]

Title changed from Issue with concatenating structured arrays to Issue with concatenating structured arrays (segmentation fault) by trac user lcampagn on 2012-09-24