search

stac-extensions / forecast

Common fields for (meteorological/weather) forecast data.
Apache License 2.0
12 stars 6 forks source link

Multi-dimensional assets and how to enode them #8

Open m-mohr opened 2 years ago

m-mohr commented 2 years ago

Based on #2: Is the extension flexible enough to encode multi-dimensional assets? What is the relation to the data cube extension (if any)? Would it make sense to allow forecast:horizon and forecast:duration to be arrays?

In #2 @cboettig also commented the following:

Also, I'm a bit unclear what if anything is assumed about the dimensionality of an individual asset. i.e. does an asset have to represent a single forecast:datetime? This makes sense for assets which are fundamentally 2d, like geotif or grib2, but a ncdf file (or hdf5, zarr, etc) can have arbitrary dimensions, i.e. forecast:datetime could be a dimension in the ncdf file, rather than serialized as metadata of separate assets.

I think these different serialization strategies should all be supported.

m-mohr commented 2 years ago

For the GEFS grib files I simply use raster:bands to encode the different characteristics.

cboettig commented 2 years ago

hmm... trying to wrap my head around this, maybe I'm not following, but this seems to overload the notion of 'band' in a way that is not very effective? For instance, the variable being measured and the ensemble number are quite different dimensions, smashing them both into 'band' feels a bit crude?

chris-little commented 2 years ago

@m-mohr @cboettig A minor point, but GRIB Edtion 2 is multidimensional. The original, obsolete, GRIB Edtion 1 was only two dimensional

And I suspect band is a legacy of geospatial, 2D, "it's a grid so must be an image of pixels from space" thinking. ;-)

In the current work in OGC on APIs, the Environmental Data Retrieval API (think 'any spatiotemporal data') are adding general dimensions, including categorical (no implicit ordering) as well as the usual (x,y,z,t).

m-mohr commented 2 years ago

The main confusing is probably that I'm not used to work with GRIB2 files and when I gdalinfo them I get a list of bands shown from GDAL. Is there a better way to inspect them with commonly used raster tooling like GDAL, rasterio etc? Maybe gdalmdiminfo?

Maybe we need to look into the STAC datacube extension then: https://github.com/stac-extensions/datacube But for that I'd need to better understand how GRIB2 files work and how I can work with them programatically.

chris-little commented 2 years ago

@m-mohr GRIB2 is basically binary data with a not-quite textual headers which specify 'templates'. E.g. one file may be a 2D quasi-horizontal array of parameters, antoher may be a 3D stack of such 2D fields over time or vertical. Some data maybe coefficients of spehical harmonic epansions of the data. The GRIB2 definition manual is complex, but see Page 31 of the PDF of the English version.

You are right that the ecosystem of software for GRIB2 is disappointingly rather sparse conisdering its importance.