extract_vars() and getvar() fail for WRF output variables DZS and ZS

[Timothy-W-Hilton]

Using either extract_vars() or getvar() to retrieve soil layer thicknesses (netCDF variables DZS and ZS) from a WRF output file fails with "KeyError: u'XLONG_Z'".

In [38]: foo = wrf.extract_vars(netCDF4.Dataset('/global/cscratch1/sd/twhilton/WRFv3.9_Sensitivity/FromScratch/WRFV3_urban/run/summen_sensitivity_urban/urban_d02_2009-06-02_12:00:00.nc'), wrf.ALL_TIMES, 'DZS')
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-38-5c5b74ce1fc9> in <module>()
----> 1 foo =     wrf.extract_vars(netCDF4.Dataset('/global/cscratch1/sd/twhilton/WRFv3.9_Sensitivity/FromScratch/WRFV3_urban/run/summen_sensitivity_urban/urban_d02_2009-06-02_12:00:00.nc'), wrf.ALL_TIMES, 'DZS')

/global/homes/t/twhilton/.conda/envs/cartopy_test_27/lib/python2.7/site-packages/wrf/util.pyc in extract_vars(wrfin, timeidx, varnames, method, squeeze, cache, meta, _key)    
   2178     return {var:_extract_var(wrfin, var, timeidx, None,
   2179                              method, squeeze, cache, meta, _key)
-> 2180             for var in varlist}
   2181
   2182

/global/homes/t/twhilton/.conda/envs/cartopy_test_27/lib/python2.7/site-packages/wrf/util.pyc in <dictcomp>((var,))
   2178     return {var:_extract_var(wrfin, var, timeidx, None,
   2179                              method, squeeze, cache, meta, _key)
-> 2180             for var in varlist}
   2181
   2182

/global/homes/t/twhilton/.conda/envs/cartopy_test_27/lib/python2.7/site-packages/wrf/util.pyc in _extract_var(wrfin, varname, timeidx, is_moving, method, squeeze, cache, meta, _key)
   2101                 is_moving = is_moving_domain(wrfin, varname, _key=_key)
   2102             result = _build_data_array(wrfin, varname, timeidx, is_moving,
-> 2103                                        multifile, _key)
   2104         else:
   2105             if not multitime:

    /global/homes/t/twhilton/.conda/envs/cartopy_test_27/lib/python2.7/site-packages/wrf/util.pyc in _build_data_array(wrfnc, varname, timeidx, is_moving_domain, is_multifile, _key)
   1212         lon_coord_vals = get_cached_item(_key, lon_coord_valkey)
   1213         if lon_coord_dims is None or lon_coord_vals is None:
-> 1214             lon_var = wrfnc.variables[lon_coord]
   1215             lon_coord_dims = lon_var.dimensions
   1216             lon_coord_vals = lon_var[:]

    KeyError: u'XLONG_Z'

The same command retrieves variable SMOIS successfully.

In [36]: foo = wrf.extract_vars(netCDF4.Dataset('/global/cscratch1/sd/twhilton/WRFv3.9_Sensitivity/FromScratch/WRFV3_urban/run/summen_sensitivity_urban/urban_d02_2009-06-02_12:00:00.nc'), wrf.ALL_TIMES, 'SMOIS')

In [37]: foo['SMOIS'].shape
Out[37]: (24, 4, 150, 114)

[bladwig1]

This is a bug and I'll have it fixed for the next release. You can get around the problem for now by setting 'meta=False' in either getvar or extract_vars. This will disable the metadata, but you'll still get the values as a regular numpy array. Thanks for reporting the issue.

[Timothy-W-Hilton]

thanks, both for the bug fix and suggesting the workaround (which had not occurred to me for some reason).

[bladwig1]

(For my own reference)

ZS and DZS do not contain the south_north or west_east dimensions, which needs to be handled. The ncdump for both is below:

float ZS(Time, soil_layers_stag) ; ZS:FieldType = 104 ; ZS:MemoryOrder = "Z " ; ZS:description = "DEPTHS OF CENTERS OF SOIL LAYERS" ; ZS:units = "m" ; ZS:stagger = "Z" ;

float DZS(Time, soil_layers_stag) ; DZS:FieldType = 104 ; DZS:MemoryOrder = "Z " ; DZS:description = "THICKNESSES OF SOIL LAYERS" ; DZS:units = "m" ; DZS:stagger = "Z" ;

The KeyError in the initial issue has already been fixed when compatibility was improved for pre-3.x WRF files (not yet released), but instead a new error is thrown:

Traceback (most recent call last): File "", line 1, in File "/Users/ladwig/miniconda2/envs/test_env/lib/python2.7/site-packages/wrf/routines.py", line 275, in getvar method, squeeze, cache, meta, _key)[varname] File "/Users/ladwig/miniconda2/envs/test_env/lib/python2.7/site-packages/wrf/util.py", line 2213, in extract_vars for var in varlist} File "/Users/ladwig/miniconda2/envs/test_env/lib/python2.7/site-packages/wrf/util.py", line 2213, in for var in varlist} File "/Users/ladwig/miniconda2/envs/test_env/lib/python2.7/site-packages/wrf/util.py", line 2136, in _extract_var multifile, _key) File "/Users/ladwig/miniconda2/envs/test_env/lib/python2.7/site-packages/wrf/util.py", line 1322, in _build_data_array attrs=attrs) File "/Users/ladwig/miniconda2/envs/test_env/lib/python2.7/site-packages/xarray/core/dataarray.py", line 224, in init coords, dims = _infer_coords_and_dims(data.shape, coords, dims) File "/Users/ladwig/miniconda2/envs/test_env/lib/python2.7/site-packages/xarray/core/dataarray.py", line 81, in _infer_coords_and_dims 'dimensions %s' % (k, v.dims, dims)) ValueError: coordinate XLONG has dimensions (u'south_north', u'west_east'), but these are not a subset of the DataArray dimensions (u'Time', u'soil_layers_stag')

After the change to support pre-3.x WRF files, if coordinate variables aren't found, it defaults to XLAT and XLONG, which isn't applicable for these two products.