cdms2 does not like a 'dimensions' variable attribute !

[jypeter]

[Go this error with cdms 3.0.0 of CDAT 8.1]

cdms2 fails with weird error messages, when it tries to write a variable that has a dimensions attribute! This could probably be detected (and see if there are other attribute names that could cause problems. I got this error yesterday and solved it by removing the attribute with

            # Do some last-minute cleanup
            if hasattr(v_out, 'dimensions'):
                # cdms2 fails when trying to write a UofT-CCSM4 variable that has the following attribute
                # clt:dimensions = "longitude latitude time" ;
                del(v_out.dimensions)

Don't why this attribute was there (it was in one of the files I got)

I reproduced the error by downloading a fx CMIP6 var (orog_fx_CESM2_piControl_r1i1p1f1_gn.nc), and using cdo to add the attribute

Creating the test file

jypeter@obelix2 - ...cdms2_bug_data - 77 >ncdump orog_fx_CESM2_piControl_r1i1p1f1_gn.nc | egrep 'coordinates|dimensions'
dimensions:
                orog:coordinates = "lat lon" ;
                orog:time_title = "No temporal dimensions ... fixed field" ;

jypeter@obelix2 - ...cdms2_bug_data - 78 >ncatted -a dimensions,orog,c,c,'lat lon' orog_fx_CESM2_piControl_r1i1p1f1_gn.nc orog_fx_CESM2_piControl_r1i1p1f1_gn_modified.nc

jypeter@obelix2 - ...cdms2_bug_data - 79 >ncdump orog_fx_CESM2_piControl_r1i1p1f1_gn_modified.nc | egrep 'coordinates|dimensions'                               dimensions:
                orog:coordinates = "lat lon" ;
                orog:time_title = "No temporal dimensions ... fixed field" ;
                orog:dimensions = "lat lon" ;
                :history = "Fri Aug  7 10:55:27 2020: ncatted -a dimensions,orog,c,c,lat lon orog_fx_CESM2_piControl_r1i1p1f1_gn.nc orog_fx_CESM2_piControl_r1i1p1f1_gn_modified.nc" ;

Reproducing the bug

(cdatm_py3) jypeter@obelix2 - ...PMIP4_Sandy - 49 >conda list | grep cdms2
cdms2                     3.0.0                    pypi_0    pypi

(cdatm_py3) jypeter@obelix2 - ...PMIP4_Sandy - 48 >python
Python 3.6.7 | packaged by conda-forge | (default, Feb 28 2019, 09:07:38)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cdms2
>>> f = cdms2.open('/home/scratch01/jypeter/cdms2_bug_data/orog_fx_CESM2_piControl_r1i1p1f1_gn_modified.nc')
>>> orog = f('orog')
>>> f.close()

>>> g = cdms2.open('/home/scratch01/jypeter/cdms2_bug_data/test_write_again.nc', 'w')
>>> g.write(orog)

[ Removed the usual "compression and no shuffling" warning ]

Traceback (most recent call last):
  File "/home/share/unix_files/cdat/miniconda3/envs/cdatm_py3/lib/python3.6/site-packages/cdms2/fvariable.py", line 157, in __setattr__
    setattr(self._obj_, name, value)
TypeError: object has read-only attributes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/share/unix_files/cdat/miniconda3/envs/cdatm_py3/lib/python3.6/site-packages/cdms2/dataset.py", line 2228, in write
    id=varid, extend=extend, fill_value=fill_value, index=index)
  File "/home/share/unix_files/cdat/miniconda3/envs/cdatm_py3/lib/python3.6/site-packages/cdms2/dataset.py", line 2119, in createVariableCopy
    setattr(newvar, attname, attval)
  File "/home/share/unix_files/cdat/miniconda3/envs/cdatm_py3/lib/python3.6/site-packages/cdms2/fvariable.py", line 161, in __setattr__
    (self.id, name, repr(value)))
cdms2.error.CDMSError: Setting orog.dimensions='lat lon'
>>>

[jasonb5]

@jypeter Thanks for reporting this.

I've reproduced the issue on our latest release and will look into it.

[jasonb5]

@jypeter After looking at the code it looks like dimensions became a reserved attribute and due to the design this prevents it from being written to a file. This behavior will change in the next major version. For the time being your workaround is the solution. I will correct the error message to indicate the real issue.

[jypeter]

Where is it specified that dimensions is a reserved attribute? Though I agree that it could make sense

I have checked all the occurrences of dimensions in the latest official version of the CF convention (1.8) and I did not find anything about dimensions as an attribute.

The Appendix A: Attributes of CF 1.8 mentions the coordinates attribute, and other similar attributes, but nothing about dimensions

Same for Appendix A: Attribute Conventions of NetCDF 4.7.4

[jasonb5]

You are correct dimensions is not a reserved attribute by CF conventions, it's an application specific one. I cannot comment on why this choice was made, as i'm not the implementing developer. The dimensions attribute seems better suited as a function and will be changed in the next major version.

[jypeter]

OK, it's something internal to cdms2! Well, it's OK as long as there is a clear error message

You may want to mention the possible workarounds:

• use del(v_out.dimensions) in the script
• usencatted from nco to rename/remove the attribute in the original file