[0.2.1] test error during build on Debian

[olebole]

This is a forward of Debian#1029244. When building the Debian package, we now observe the following test failure:

_______________________ test_ms_tables[SPECTRAL_WINDOW] ________________________
tablename = 'SPECTRAL_WINDOW'

    @pytest.mark.parametrize('tablename', ('.', […])
    def test_ms_tables(tablename):

        table_filename = os.path.join(DATA, 'simple.ms', tablename)

        # Concatenation issue as arrays change shape half way through
        if tablename == 'SYSPOWER':
            pytest.xfail()

>       tt = [Table.read(table_filename, data_desc_id=0),
              Table.read(table_filename, data_desc_id=1)]

casa_formats_io/casa_low_level_io/tests/test_casa_low_level_io.py:215: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3/dist-packages/astropy/table/connect.py:62: in __call__
    out = self.registry.read(cls, *args, **kwargs)
/usr/lib/python3/dist-packages/astropy/io/registry/core.py:219: in read
    data = reader(*args, **kwargs)
casa_formats_io/table_reader.py:18: in read_casa_table
    return table.as_astropy_table(data_desc_id=data_desc_id)
casa_formats_io/casa_low_level_io/table.py:365: in as_astropy_table
    coldata = dm.read_column(self._filename, seqnr, self.column_set.columns[colindex], coldesc[colindex], colindex_in_dm)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = StandardStMan{'_fileobj': <casa_formats_io.casa_low_level_io.core.EndianAwareFileHandle object at 0x7f7a1d224e90>,
 'b...s': 6,
 'number_of_bucket_for_index': 1,
 'number_of_buckets': 8,
 'number_of_free_buckets': 0,
 'persistent_cache': 2}
filename = '/build/casa-formats-io-0.2.1/.pybuild/cpython3_3.11_casa-formats-io/build/casa_formats_io/casa_low_level_io/tests/data/simple.ms/SPECTRAL_WINDOW'
seqnr = 0
column = PlainColumn{'data': ArrayColumnData{'seqnr': 0, 'shape': (), 'version': 1},
 'name': 'CHAN_FREQ'}
coldesc = ColumnDesc{'comment': 'Center frequencies for each channel in the data matrix',
 'data_manager_group': 'StandardStMan'...dim': 1,
 'option': 0,
 'shape': array([], dtype=int64),
 'stype': 'ArrayColumnDesc<double  ',
 'value_type': 'double'}
colindex_in_dm = 1

    def read_column(self, filename, seqnr, column, coldesc, colindex_in_dm):
        […]

        for bucket_id in index.bucket_number.elements:
            […]
            if coldesc.value_type == 'string':
                […]
            else:
                if coldesc.is_direct or 'Scalar' in coldesc.stype:
                    data.append(read_as_numpy_array(f, coldesc.value_type, rows_in_bucket[bucket_id] * nelements, shape=(-1,) + shape[::-1]))
                else:
                    values = []
                    for irow in range(rows_in_bucket[bucket_id]):
                        offset = read_int64(f)
                        fi.seek(offset)
                        ndim = read_int32(fi)
                        subshape = []
                        for idim in range(ndim):
                            subshape.append(read_int32(fi))
                        size = int(np.product(subshape))
                        values.append(read_as_numpy_array(fi, coldesc.value_type, size, shape=subshape[::-1]))
>                   data.append(np.array(values))
E                   ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.

casa_formats_io/casa_low_level_io/data_managers/standard.py:208: ValueError

This is probably caused by the update of numpy to 1.24. The full build log is in the Debian bug.

[olebole]

The reason here is that numpy now requires to specify dtype=object for this kind of array; see NEP 34. With

--- a/casa_formats_io/casa_low_level_io/data_managers/standard.py
+++ b/casa_formats_io/casa_low_level_io/data_managers/standard.py
@@ -205,7 +205,7 @@ class StandardStMan(BaseCasaObject):
                             subshape.append(read_int32(fi))
                         size = int(np.product(subshape))
                         values.append(read_as_numpy_array(fi, coldesc.value_type, size, shape=subshape[::-1]))
-                    data.append(np.array(values))
+                    data.append(np.array(values, dtype=object))
         if data:
             if data[0].ndim > 1:
                 return np.vstack(data)

it works for me. However, since I can't run the full test (casatools is not available in Debian), it may be that there are more of these; that's why I didn't just makr a PR.

[astrofrog]

I'll try and look into this shortly