Unidata / siphon

Siphon - A collection of Python utilities for retrieving atmospheric and oceanic data from remote sources, focusing on being able to retrieve data from Unidata data technologies, such as the THREDDS data server.
https://unidata.github.io/siphon
BSD 3-Clause "New" or "Revised" License
211 stars 74 forks source link

Writing out a siphon'd xarray dataset as a netCDF #762

Open ThomasMGeo opened 3 weeks ago

ThomasMGeo commented 3 weeks ago

Hello! I am trying to write out a netCDF file from a siphon'd xarray dataset. Writing out to Zarr works fine. This may be in the wrong spot (see image).

That is the input:

catUrl = "https://thredds.ucar.edu/thredds/catalog/grib/NCEP/GFS/Global_onedeg_ana/catalog.xml";
datasetName = "Full Collection Dataset";

catalog = TDSCatalog(catUrl)

ds2 = catalog.datasets[datasetName].remote_access(use_xarray=True)
print('ds is a:', type(ds2))

and writing it out:

# Write the second dataset (ds2) to NetCDF format and print the output path
ds2.to_netcdf(paths['ds2'])
print(f"Second dataset written to {paths['ds2']} in NetCDF format.")

I just wanted to write out a netCDF file, I didn't modify the file to my knowledge. Happier to share a more in depth workflow. The error I get is below:

RuntimeError: NetCDF: Name contains illegal characters

siphon version 0.9 8tg06w

ThomasMGeo commented 3 weeks ago

Notebook that has more of the workflow: https://github.com/Unidata/MLscratchpad/blob/main/BlogNotebooks/netCDF_vs_Zarr_filestoage.ipynb

tdrwenski commented 1 week ago

I reproduced this and did notice that

print(ds2.dims)

returns

Frozen({'sigma_layer': 4, '': 2, 'pressure_difference_layer': 1, 'pressure_difference_layer1': 1, 'time': 1, 'isobaric': 41, 'lat': 181, 'lon': 360, 'hybrid': 1, 'isobaric1': 22, 'potential_vorticity_surface': 2, 'sigma': 1, 'altitude_above_msl': 3, 'height_above_ground': 2, 'height_above_ground1': 6})

I think possibly the "Name contains illegal characters" could be referring to a dimension of size 2 with an empty name. Not sure why that dimension has no name when reading in with xarray but is not present when reading with Opendap though.