radio-astro-tools / casa-formats-io

Code to handle I/O from/to data in CASA format
Other
10 stars 7 forks source link

Test broke with new casatools 6.4.3.8 version #39

Closed astrofrog closed 7 months ago

astrofrog commented 2 years ago

The following test failure is new with casatools 6.4.3.8:

______________________________________________________________________ test_generic_table_read ______________________________________________________________________

tmp_path = PosixPath('/tmp/pytest-of-root/pytest-0/test_generic_table_read0')

    @pytest.mark.openfiles_ignore
    @pytest.mark.skipif('not CASATOOLS_INSTALLED')
    def test_generic_table_read(tmp_path):

        # NOTE: for now, this doesn't check that we can read the data - just
        # the metadata about the table.

        filename_fits = str(tmp_path / 'generic.fits')
        filename_casa = str(tmp_path / 'generic.image')

        N = 120

        t = Table()
        t['short'] = np.arange(N, dtype=np.int16)
        t['ushort'] = np.arange(N, dtype=np.uint16)
        t['int'] = np.arange(N, dtype=np.int32)
        t['uint'] = np.arange(N, dtype=np.uint32)
        t['float'] = np.arange(N, dtype=np.float32)
        t['double'] = np.arange(N, dtype=np.float64)
        t['complex'] = np.array([1 + 2j, 3.3 + 8.2j, -1.2 - 4.2j] * (N // 3), dtype=np.complex64)
        t['dcomplex'] = np.array([3.33 + 4.22j, 3.3 + 8.2j, -1.2 - 4.2j] * (N // 3), dtype=np.complex128)
        t['str'] = np.array(['reading', 'casa', 'images'] * (N // 3))

        # Repeat this at the end to make sure we correctly finished reading
        # the complex column metadata
        t['int2'] = np.arange(N, dtype=np.int32)

        t.write(filename_fits)

        tb = table()
        tb.fromfits(filename_casa, filename_fits)
        tb.close()

        # Use the arrays in the table to also generate keywords of various types
        keywords = {'scalars': {}, 'arrays': {}}
        for name in t.colnames:
            keywords['scalars']['s_' + name] = t[name][0]
            keywords['arrays']['a_' + name] = t[name]

        tb.open(filename_casa)
>       tb.putkeywords(keywords)

casa_formats_io/casa_low_level_io/tests/test_casa_low_level_io.py:126: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.6/dist-packages/casatools/table.py:972: in putkeywords
    return self._swigobj.putkeywords(_dict_ec(value))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <casatools.__casac__.table.table; proxy of <Swig Object of type 'casac::table *' at 0x7f7a353d0db0> >
args = ({b'arrays': {b'a_complex': <Column name='complex' dtype='complex64' length=120>
     (1+2j)
 (3.3+8.2j)
(-1.2-4.2j)
 ...119.0, ...}, b'scalars': {b's_complex': (1+2j), b's_dcomplex': (3.33+4.22j), b's_double': 0.0, b's_float': 0.0, ...}},)
kwargs = {}

    def putkeywords(self, *args, **kwargs) -> "bool":
        """
        putkeywords(self, _value) -> bool

        Summary:
            !!!BROKEN!!! put multiple table keywords

        Description:

        Put multiple table keywords. All fields in the given record are put
        as table keywords. The value of each field can be a scalar or
        an array of any type or it can be a record.
        It is also possible to define a keyword holding a subtable.
        This can be done by giving the keyword a string value consisting of
        the subtable name prefixed by 'Table: '.

        Input Parameters:
            value                     Record of keyword=value pairs

        Example:

        tb.open('3C273XC1.MS', nomodify=False)
        kw=tb.getkeywords()
        print kw['MS_VERSION']
        # 2.0
        kw['MS_VERSION']=2.1
        tb.putkeywords(kw)
        # !!!BROKEN. Keywords containing float are not handled properly!!!
        tb.flush()
        # True

        --------------------------------------------------------------------------------

        """
>       return _table.table_putkeywords(self, *args, **kwargs)
E       RuntimeError: Table::rwKeywordSet cannot be used: table /tmp/pytest-of-root/pytest-0/test_generic_table_read0/generic.image is not writable

/usr/local/lib/python3.6/dist-packages/casatools/__casac__/table.py:2689: RuntimeError

It did not fail with the previous version - 6.4.3.5

I'm not sure what to to do about this or how to report this upstream - but the # !!!BROKEN. Keywords containing float are not handled properly!!! suggests this might be a real issue.

For now we can pin to <6.4.3.8 for testing.

keflavich commented 2 years ago

@preshanth suggested we switch our CASA testing to 3.8.

preshanth commented 2 years ago

If that is still broken then I will take it up with the devs. The move to 3.8 sort of removes the need for old Ubuntu versions to run it properly. Removes that really annoying libgfortran.so.6 requirement.

astrofrog commented 2 years ago

Ok cool - we could definitely try that out!

astrofrog commented 7 months ago

This is because we now need to set nomodify=False when opening a table to edit.