Bigedian data seems don't convert to the compiler.

[singledoggy]

So we may get the error:

Big-endian buffer not supported on little-endian compiler

If we use open_CtlDataset(ctl_path) as xarray data and use it in pandas or someting else.

And xarray will Automatically decode netCDF data to native endianness, so we may use .to_netcdf method to convert it and reload to solve this error. So I'm not sure if this feathure should be added to this repository or people can just use the function of xarray. If any one meet the same mistake, hope this can help.

[miniufo]

Could you please show some code snippet so that I can understand the problem much better. I am not sure if it is a problem with xgrads. But since xgrads package is built on xarray, we may also get some errors with xarray.

[singledoggy]

Here is my snippet, and it's hard for me to reproduce it in just pandas (I use geopandas), the results of load the ctl as xarraydata directly and convert it to .nc and reload is different. Maybe xarray convert .nc file to native endiannesss, and xgrads don't. Here are another link about it in the doc of pandas. I don't know if it's necessary to deal with this issue. Because xarray has solved it in the link I show above. The users can output the data and reload it.

# %%
import cartopy.crs as ccrs
import geopandas as gpd
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import xarray as xr
from xgrads import CtlDescriptor, get_data_projection, open_CtlDataset

ctl_path = '/Users/sam/xgrads/ctls/test8_2.ctl'
# This change the endianness
# open_CtlDataset(ctl_path).to_netcdf(  'naq_data.nc')
# naq_data = xr.open_dataset('naq_data.nc')

naq_data= open_CtlDataset(ctl_path)

ctl = CtlDescriptor(file=ctl_path)
crs = get_data_projection(ctl)

city = gpd.read_file(
    r'city.shp')
city = city.to_crs(crs.proj4_init)

our_source = naq_data.to_dataframe().reset_index()
gdf = gpd.GeoDataFrame(
    our_source, geometry=gpd.points_from_xy(our_source.lat, our_source.lon))
gdf.crs = crs.proj4_init

grid_sta = gpd.overlay(gdf, city, how='intersection')

[miniufo]

Actually, xarray allows you to change the endianess of the data. For example, var = xr.open_dataset('naq_data').xxx_var.astype('>f4') changes the variable xxx_var to big_endian and var = xr.open_dataset('naq_data').xxx_var.astype('<f4') changes it to little_endian.

You can also change the whole dataset to big_endian before writting to netcdf file as: open_CtlDataset(ctl_path).astype('>f4').to_netcdf( 'naq_data.nc')

Hope this helps.

[singledoggy]

It really helps. But xarray seems has do .astype automatically so I had never notice this feature. So I'm just wondering if it's necessary for this package to convert endianess by default. I'm not sure about the pros and cons.

Thanks a lot.

[miniufo]

The philosophy of xgrads is to parse and load binary data into xarray dataset. The endianness is generally specified by ctl files. If not, the endianness is determined defaultly by the platform one uses. Since astype is convenient to change the endianness, I guess we don't need to add any similar functionality to xgrads.

Glad you are interested in this package.