andersy005
commented
2 months ago cf_xarrayseems confused aboutevolv2k
In [9]: url = "https://ncsa.osn.xsede.org/Pangeo/pangeo-forge/pangeo-forge/eVolv2k-feedstock/eVolv2k.zarr"
In [10]: ds = xr.open_zarr(url)
In [11]: cf_xarray.accessor._get_axis_coord(ds, "Y")
Out[11]: []
In [12]: ds
Out[12]:
<xarray.Dataset> Size: 11kB
Dimensions: (nerup: 271, time: 271)
Coordinates:
* time (time) object 2kB -0490-01-01 00:00:00 ... 1890-01-01 00:00:00
Dimensions without coordinates: nerup
Data variables:
hemi (nerup) float64 2kB dask.array<chunksize=(271,), meta=np.ndarray>
lat (nerup) float64 2kB dask.array<chunksize=(271,), meta=np.ndarray>
sigma_ssi (nerup) float64 2kB dask.array<chunksize=(271,), meta=np.ndarray>
ssi (nerup) float64 2kB dask.array<chunksize=(271,), meta=np.ndarray>
Attributes:
comment: Minor update from v 2.0, includes reassignment of eruption regi...
history: Created Mon Oct 16 10:31:15 2017
source: Toohey and Sigl, 2017
title: Ice core-inferred volcanic stratospheric sulfur injection from ...
In [14]: ds.cf.axes
Out[14]: {}In [15]: url = "https://ncsa.osn.xsede.org/Pangeo/pangeo-forge/pangeo-forge/LMRv2p1_MCruns_ensemble_gridded-feedstock/LMRv2p1_MCruns_ensemble_gridded.zarr"
In [16]: ds = xr.open_zarr(url)
In [17]: ds
Out[17]:
<xarray.Dataset> Size: 37GB
Dimensions: (time: 2001, MCrun: 20, lat: 91, lon: 180)
Coordinates:
* lat (lat) float32 364B -90.0 -88.0 -86.0 -84.0 ... 86.0 88.0 90.0
* lon (lon) float32 720B 0.0 2.0 4.0 6.0 ... 354.0 356.0 358.0
* time (time) object 16kB 0000-01-01 00:00:00 ... 2000-01-01 00:0...
Dimensions without coordinates: MCrun
Data variables: (12/14)
air_mean (time, MCrun, lat, lon) float32 3GB dask.array<chunksize=(1, 20, 91, 180), meta=np.ndarray>
air_spread (time, MCrun, lat, lon) float32 3GB dask.array<chunksize=(1, 20, 91, 180), meta=np.ndarray>
hgt500_mean (time, MCrun, lat, lon) float32 3GB dask.array<chunksize=(1, 20, 91, 180), meta=np.ndarray>
hgt500_spread (time, MCrun, lat, lon) float32 3GB dask.array<chunksize=(1, 20, 91, 180), meta=np.ndarray>
pdsi_mean (time, MCrun, lat, lon) float32 3GB dask.array<chunksize=(1, 20, 91, 180), meta=np.ndarray>
pdsi_spread (time, MCrun, lat, lon) float32 3GB dask.array<chunksize=(1, 20, 91, 180), meta=np.ndarray>
... ...
prate_mean (time, MCrun, lat, lon) float32 3GB dask.array<chunksize=(1, 20, 91, 180), meta=np.ndarray>
prate_spread (time, MCrun, lat, lon) float32 3GB dask.array<chunksize=(1, 20, 91, 180), meta=np.ndarray>
prmsl_mean (time, MCrun, lat, lon) float32 3GB dask.array<chunksize=(1, 20, 91, 180), meta=np.ndarray>
prmsl_spread (time, MCrun, lat, lon) float32 3GB dask.array<chunksize=(1, 20, 91, 180), meta=np.ndarray>
sst_mean (time, MCrun, lat, lon) float32 3GB dask.array<chunksize=(1, 20, 91, 180), meta=np.ndarray>
sst_spread (time, MCrun, lat, lon) float32 3GB dask.array<chunksize=(1, 20, 91, 180), meta=np.ndarray>
Attributes:
comment: File contains ensemble spread values for each ...
description: Last Millennium Reanalysis climate field recon...
experiment: productionFinal2_gisgpcc_ccms4_LMRdbv1.1.0_z500
pangeo-forge:inputs_hash: c148739ca23233f5121dd7f6ac70826b68f8831d19191b...
pangeo-forge:recipe_hash: aa66a32d990e984111f664c3a5bdd5326edcfcf20a588d...
pangeo-forge:version: 0.9.2
In [19]: ds.cf.axes
Out[19]: {'X': ['lon'], 'Y': ['lat'], 'T': ['time']}- this dataset fails the validation check in ncview-js due to the
MCrunbeing a dimension without zarray associated with it.
const finalVariables = dimensionalCheck.filter((d) => {
const dims = metadata.metadata[`${prefix}${d}/.zattrs`]['_ARRAY_DIMENSIONS'];
const allDimsExist = dims.every((dim) => {
const exists = metadata.metadata[`${prefix}${dim}/.zarray`];
if (!exists) {
console.log(`Missing zarray for dimension '${dim}' in variable '${d}'`);
}
return exists;
});
return allDimsExist;
});
console.log(finalVariables);
VM1759:6 Missing zarray for dimension 'MCrun' in variable 'air_mean'
VM1759:6 Missing zarray for dimension 'MCrun' in variable 'air_spread'
VM1759:6 Missing zarray for dimension 'MCrun' in variable 'hgt500_mean'
VM1759:6 Missing zarray for dimension 'MCrun' in variable 'hgt500_spread'
VM1759:6 Missing zarray for dimension 'MCrun' in variable 'pdsi_mean'
VM1759:6 Missing zarray for dimension 'MCrun' in variable 'pdsi_spread'
VM1759:6 Missing zarray for dimension 'MCrun' in variable 'pr_wtr_mean'
VM1759:6 Missing zarray for dimension 'MCrun' in variable 'pr_wtr_spread'
VM1759:6 Missing zarray for dimension 'MCrun' in variable 'prate_mean'
VM1759:6 Missing zarray for dimension 'MCrun' in variable 'prate_spread'
VM1759:6 Missing zarray for dimension 'MCrun' in variable 'prmsl_mean'
VM1759:6 Missing zarray for dimension 'MCrun' in variable 'prmsl_spread'
VM1759:6 Missing zarray for dimension 'MCrun' in variable 'sst_mean'
VM1759:6 Missing zarray for dimension 'MCrun' in variable 'sst_spread'
VM1759:12
[]- the following dataset fails and this is the message i get on the front-end:
Cannot read properties of undefined (reading 'X')not sure where this property is being accessed and causing this issue
In [23]: ds = xr.open_dataset(url, engine='zarr', chunks={}, consolidated=True)
In [24]: ds.cf.axes
Out[24]: {'X': ['longitude'], 'Y': ['latitude'], 'T': ['time']}
katamartin
norlandrhagen
Cc @katamartin