'TransientCurveGrid' object has no attribute 'getWeights'

[yyplanton]

Hi, I am trying to use the cdutil.averager function on cmip5 datasets. Out of 43 models tested, 6 (BBC and GFDL models) return the same error when averaging oceanic variables (like 'tos') I am using Transient Variables (MV2s) so weights must be generated: Traceback (most recent call last): File "test.py", line 15, in d_ave_xy = CDUTILaverager(tos, axis='xy') File "/usr/local/uvcdat/2.4.0/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/genutil/averager.py", line 1078, in averager filled_wtoptions = __check_weightoptions(V, axis, weights) File "/usr/local/uvcdat/2.4.0/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/genutil/averager.py", line 359, in __check_weightoptions weightoptions = area_weights(x,axisoptions) File "/usr/local/uvcdat/2.4.0/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/genutil/averager.py", line 159, in area_weights latwts, lonwts = dsgr.getWeights() AttributeError: 'TransientCurveGrid' object has no attribute 'getWeights'

Here is the grid:

tos.getGrid() <TransientCurveGrid, id: grid_1, shape: (232, 360)>

I do not have the same problem when using other TransientCurveGrid models like CNRM-CM5

Models returning errors: bcc-csm1-1-m, bcc-csm1-1, GFDL-CM2p1, GFDL-CM3, GFDL-ESM2G, GFDL-ESM2M A common point between these models is that they use GFDL’s Modular Ocean Model (MOM) or Generalized Ocean Layer Dynamics (GOLD) (only GFDL-ESM2G).

Is there a problem with GFDL's ocean models? Is this error known?

Thanks Yann

[jypeter]

I'm not even sure cdutil.averager can handle irregular grids. @doutriaux1 ?

It's probably better to use the areacellovariable and compute the weights yourself, in that case

[dnadeau4]

@doutriaux1 can you help @yyplanton with this question.

[lee1043]

@gleckler1 @durack1 Any comment welcome. Maybe areacello variable should be adapted? Or maybe using esmf regrid to interpolatetos to regular grid as pre-process would be helpful?

[gleckler1]

@yyplanton GFDL did not use CMOR in CMIP5. While they were able to mimic it very well, we have run across subtle discrepancies. Can you show us your CDAT code, i.e., what is in your function CDUTILaverager?

[gleckler1]

@yyplanton @jypeter You are right JY, use MV.multiply(areacello, tos) ... cdutil does not yet work on native grids

[doutriaux1]

@yyplanton cdutil.averager only works with rectinlinear grids, not oceanic grids. There's no plans to immediately adapt it, although it's a good idea. But you would need to somehow provide weights. @dnadeau4 could we use esmf to generate the weights? @yyplanton one way I have seen avergaing done in the past is to *trick a regridder, create a target grid with one cell only covering the whole globe, then regird your data to this grid and voila

globalGrid = cdms2.createGlobalMeanGrid(cdms2.createGaussianGrid(128))
mean = data.regrid(globalGrid)

[yyplanton]

Ok, thanks for the answers. @doutriaux1, as I said, cdutil.averager works for CNRM-CM5 and some ther models using TransientCurveGrid. In the case of CNRM-CM5, the oceanic grid is a NEMO orca1 grid. But I am working on the equatorial Pacific and in this region the NEMO orca1 grid is very close to a rectinlinear grid.

[jypeter]

I'm surprised that it works with a TransientCurveGrid! @doutriaux1 @dnadeau4 does cdutil.averagercheck that it can safely operate? Maybe it should raise an exception when that's not the case. I have found an old issue about that https://github.com/UV-CDAT/uvcdat/issues/1464

I have just remembered an old related question: in the case when cdutil.averageris passed a regular grid, that has no bounds (because there were no bounds in the input file) and setAutoBoundsis off, does averager now print a nice error message for the user, or does it still throw a scary (for beginning users!) traceback?

[doutriaux1]

@yyplanton I am surprised it works for curvilinear grid, can you send me the line you use. I suspect what you're actually doing is a straight average of the cell, not an area weighted average.

[lee1043]

It is interesting to see that CNRM-CM5 is working with `cdutil.averager'.

For curiosity, I simply visualized tos field itself by using below script.

import cdms2, cdutil, vcs

modpath = '/work/lee1043/ESGF/xmls/cmip5/historical/mo/VAR/cmip5.MOD.historical.r1i1p1.mo.VAR.xml'
modname = 'CNRM-CM5'
varname = 'tos'

filename = modpath.replace('MOD',modname).replace('VAR',varname)
print filename

f = cdms2.open(filename)
d = f(varname)

x = vcs.init()
x.plot(d)
x.png('test_'+modname+'_'+varname+'.png')

VCS plot (CDAT 3.0 beta version):

Ncview:

@doutriaux1 Why VCS plot show zonal band of missing values? Other question would be, (sorry not related to this issue though) does CDAT 3.0 use boxfill with discrete color as default?

[lee1043]

@yyplanton not sure how CNRM-CM5 was successful in ENSO metrics code, because when I test as below it does not work. Are you using areacello to get weighting?

Code:

import cdms2, cdutil, vcs

modpath = '/work/lee1043/ESGF/xmls/cmip5/historical/mo/VAR/cmip5.MOD.historical.r1i1p1.mo.VAR.xml'
modname = 'CNRM-CM5'
varname = 'tos'

filename = modpath.replace('MOD',modname).replace('VAR',varname)
print filename

f = cdms2.open(filename)
d = f(varname)

a = cdutil.averager(d,axis='12')
print a

Error:

(cdat.3.0.beta-2) -bash-4.1$ pi check_data_CNRM-CM5.py 
/work/lee1043/ESGF/xmls/cmip5/historical/mo/tos/cmip5.CNRM-CM5.historical.r1i1p1.mo.tos.xml
Traceback (most recent call last):
  File "check_data_CNRM-CM5.py", line 13, in <module>
    a = cdutil.averager(d,axis='12')
  File "/export_backup/lee1043/anaconda2/envs/cdat.3.0.beta-2/lib/python2.7/site-packages/genutil/averager.py", line 1264, in averager
    filled_wtoptions = __check_weightoptions(V, axis, weights)
  File "/export_backup/lee1043/anaconda2/envs/cdat.3.0.beta-2/lib/python2.7/site-packages/genutil/averager.py", line 481, in __check_weightoptions
    weightoptions = area_weights(x, axisoptions)
  File "/export_backup/lee1043/anaconda2/envs/cdat.3.0.beta-2/lib/python2.7/site-packages/genutil/averager.py", line 282, in area_weights
    wt = __myGetAxisWeights(ds, 0, axisoptions)
  File "/export_backup/lee1043/anaconda2/envs/cdat.3.0.beta-2/lib/python2.7/site-packages/genutil/averager.py", line 83, in __myGetAxisWeights
    'Bounds not available to compute weights on dimension: ' + ax.id)
genutil.averager.AveragerError: ('B', 'o', 'u', 'n', 'd', 's', ' ', 'n', 'o', 't', ' ', 'a', 'v', 'a', 'i', 'l', 'a', 'b', 'l', 'e', ' ', 't', 'o', ' ', 'c', 'o', 'm', 'p', 'u', 't', 'e', ' ', 'w', 'e', 'i', 'g', 'h', 't', 's', ' ', 'o', 'n', ' ', 'd', 'i', 'm', 'e', 'n', 's', 'i', 'o', 'n', ':', ' ', 'j')

Axes checking

>>> d.getAxisList()
[   id: time
   Designated a time axis.
   units:  days since 1850-01-01 00:00:00
   Length: 1872
   First:  15.5
   Last:   56962.5
   Other axis attributes:
      calendar: gregorian
      axis: T
      realtopology: linear
   Python id:  0x7f205b51f310
,    id: j
   units:  1
   Length: 292
   First:  1
   Last:   292
   Other axis attributes:
      long_name: cell index along second dimension
      realtopology: linear
   Python id:  0x7f205b51f890
,    id: i
   units:  1
   Length: 362
   First:  1
   Last:   362
   Other axis attributes:
      long_name: cell index along first dimension
      realtopology: linear
   Python id:  0x7f205b51f0d0

[jypeter]

@lee1043 the orca grid has redundant points on the edges that could cause the zonal band. See how I had to massage the data in https://github.com/UV-CDAT/uvcdat/issues/1465#issuecomment-133473257, in order to plot orca data

[lee1043]

Any update for this? @yyplanton and @lee1043 is revisiting this issue with zos variable.. we don't have the problem with CNRM for now but have problem with MIROC6 and few others.

[durack1]

@yyplanton @lee1043 thanks for pushing on this, there are a couple of open bugs that are hitting similar issues with ocean grids, so when this issue is closed these will also likely be closed: https://github.com/CDAT/cdat/issues/1465 https://github.com/CDAT/cdat/issues/1848

And just for completeness a couple of closed issues: https://github.com/CDAT/cdat/issues/1590 https://github.com/CDAT/cdat/issues/1707 https://github.com/CDAT/cdat/issues/1832 https://github.com/CDAT/cdat/issues/1833

[lee1043]

@durack1 thanks for the information. I will keep on eyes on those issues as well.

[github-actions[bot]]

Marking issue as stale, since there has been no activity in 30 days.

Unless the issue is updated or the 'stale' tag is removed, this issue will be closed in 7 days.

[durack1]

This is another grid issue, just like https://github.com/CDAT/genutil/issues/32, https://github.com/CDAT/genutil/issues/6 and https://github.com/CDAT/genutil/issues/5, there are likely more examples, such as https://github.com/CDAT/cdat/issues/1833, https://github.com/CDAT/cdutil/issues/23 etc etc

[github-actions[bot]]

Marking issue as stale, since there has been no activity in 30 days.

Unless the issue is updated or the 'stale' tag is removed, this issue will be closed in 7 days.