NCAR / wrf-python

A collection of diagnostic and interpolation routines for use with output from the Weather Research and Forecasting (WRF-ARW) Model.
https://wrf-python.readthedocs.io
Apache License 2.0
409 stars 154 forks source link

use to_netcdf() : save value to netcdf file #91

Open xigrug opened 5 years ago

xigrug commented 5 years ago

Here is a value:

<xarray.DataArray 'height_interp' (Time: 24, south_north: 282, west_east: 540)>

array([[[146.38087, 146.36438, ..., 150.47563, 150.22495], [146.31306, 146.33267, ..., 150.4368 , 150.4107 ], ..., [144.736 , 144.68318, ..., 147.91237, 147.94415], [146.47084, 144.24289, ..., 147.98172, 148.00264]], [[146.63193, 146.64368, ..., 150.74966, 150.60242], [146.59207, 146.61151, ..., 150.64212, 150.65385], ..., [145.27626, 145.13568, ..., 147.51866, 147.48692], [145.96034, 144.85298, ..., 147.57097, 147.56674]], ..., [[146.64447, 146.59451, ..., 149.74435, 149.65564], [146.64937, 146.62129, ..., 149.73277, 149.69064], ..., [145.3436 , 145.35204, ..., 144.83829, 144.81752], [145.42519, 145.33458, ..., 144.85371, 144.82965]], ..., [[146.03555, 146.10979, ..., 149.63452, 149.66908], [146.15005, 146.14072, ..., 149.64066, 149.6417 ], ..., [145.02827, 144.96996, ..., 145.09335, 145.08134], [144.8737 , 144.95973, ..., 145.12323, 145.0956 ]]], dtype=float32) Coordinates: XLONG (south_north, west_east) float32 79.74786 79.82733 ... 134.49786 XLAT (south_north, west_east) float32 14.181648 14.207611 ... 39.589397

  • Time (Time) datetime64[ns] 2013-06-01 ... 2013-06-01T23:00:00 datetime (Time) datetime64[ns] 2013-06-01 ... 2013-06-01T23:00:00 level int64 850 Dimensions without coordinates: south_north, west_east Attributes: FieldType: 104 units: dm stagger:
    coordinates: XLONG XLAT projection: LambertConformal(stand_lon=105.81999969482422, moad_cen_l... missing_value: 9.969209968386869e+36 _FillValue: 9.969209968386869e+36 vert_units: hPa

hgt_850.to_netcdf("hgt_850.nc")

TypeError Traceback (most recent call last)

in ----> 1 hgt_850.to_netcdf("hgt_850.nc") ~/soft/anaconda2/envs/py3/lib/python3.6/site-packages/xarray/core/dataarray.py in to_netcdf(self, *args, **kwargs) 1759 dataset = self.to_dataset() 1760 -> 1761 return dataset.to_netcdf(*args, **kwargs) 1762 1763 def to_dict(self, data=True): ~/soft/anaconda2/envs/py3/lib/python3.6/site-packages/xarray/core/dataset.py in to_netcdf(self, path, mode, format, group, engine, encoding, unlimited_dims, compute) 1321 engine=engine, encoding=encoding, 1322 unlimited_dims=unlimited_dims, -> 1323 compute=compute) 1324 1325 def to_zarr(self, store=None, mode='w-', synchronizer=None, group=None, ~/soft/anaconda2/envs/py3/lib/python3.6/site-packages/xarray/backends/api.py in to_netcdf(dataset, path_or_file, mode, format, group, engine, encoding, unlimited_dims, compute, multifile) 769 # validate Dataset keys, DataArray names, and attr keys/values 770 _validate_dataset_names(dataset) --> 771 _validate_attrs(dataset) 772 773 try: ~/soft/anaconda2/envs/py3/lib/python3.6/site-packages/xarray/backends/api.py in _validate_attrs(dataset) 167 for variable in dataset.variables.values(): 168 for k, v in variable.attrs.items(): --> 169 check_attr(k, v) 170 171 ~/soft/anaconda2/envs/py3/lib/python3.6/site-packages/xarray/backends/api.py in check_attr(name, value) 158 'a string, an ndarray or a list/tuple of ' 159 'numbers/strings for serialization to netCDF ' --> 160 'files'.format(value)) 161 162 # Check attrs on the dataset itself **TypeError: Invalid value for attr: LambertConformal(stand_lon=105.81999969482422, moad_cen_lat=30.45999526977539, truelat1=30.0, truelat2=60.0, pole_lat=90.0, pole_lon=0.0) must be a number, a string, an ndarray or a list/tuple of numbers/strings for serialization to netCDF files**
mhaselsteiner commented 5 years ago

I also encounter the issue that the way the projection ist stored as an objects breaks the xarray.DataArray method to_netcdf()

I did a work around using the following function:

def write_xarray_to_netcdf(xarray_array, output_path,mode='w', format='NETCDF4', group=None, engine=None,
                           encoding=None):
    """writes and xarray in a netcdf format outputfile
    Uses the xarray typical for wrf-python. The projection objects are transformed into strings
    to be able to use them as netcdf attributes
    :param xarray_array: xarray.DataArray
    :param output_path: str
    :param format: 'NETCDF4', 'NETCDF4_CLASSIC', 'NETCDF3_64BIT' or 'NETCDF3_CLASSIC'
                    default: 'NETCDF4'
    :param group: str, default None
    :param engine: 'netcdf4', 'scipy' or 'h5netcdf'
    :param encoding: dict, default: None
    """
    xarray_array_out = xarray_array.copy(deep=True)
    # coordinates are extracted from variable
    del xarray_array_out.attrs['coordinates']
    # wrf-python projection object cannot be processed
    xarray_array_out.attrs['projection'] = str(xarray_array_out.attrs['projection'])

    xarray_array_out.to_netcdf(path=output_path, mode=mode, format=format, group=group,
                               engine=engine,
                               encoding=encoding)

There is also an issue considering wrf data to netcdf at the xarray package.
However, I think writing to netcdf is common for wrf user and it might be worth adding a function to this package or overriding the xarray method. This might be related to the more gerneal issue #16 . If there already is a simpler way to do this with this package, I'd be grateful to know

thaolinhtran commented 2 years ago

Hi all, I would like to plot figures after importing data from the netcdf file written. I was wondering if it is possible to convert back the attrs information from string to the original wrf.projection object format? Any suggestion would be much appreciated.

DWesl commented 2 months ago

Is it possible to convert back the attrs information from string to the original wrf.projection object format?

Options:

  1. (from the function above): Perform the changes on a deep copy of the dataset, so the original dataset is unchanged
  2. Save the projection object before saving to netCDF, then restore from that variable:
    proj = ds.attrs["projection"]
    ds.attrs["projection"] = str(proj)
    ds.to_netcdf(...)
    ds.attrs["projection"] = proj