search

stactools-packages / noaa-nwm

Other
1 stars 0 forks source link

Couldn't find this package on pypi at advertised URL, install from source? #2

Open rsignell opened 1 month ago

rsignell commented 1 month ago

I read with interest the kerchunk feature of xstac: https://github.com/stac-utils/xstac?tab=readme-ov-file#kerchunk-support but when I looked for how to enable the example, I could not find the referenced package on pypi: https://pypi.org/project/stactools-noaa-nwm/

I tried searching pypi for "nwm" and didn't see any likely looking packages either.

Do we need to install from source, or is this package deprecated in favor of some other package?

TomAugspurger commented 1 month ago

The PyPI link probably came from the repo template. https://github.com/stactools-packages/noaa-nwm/pull/1 has the initial prototype for this package, but it wasn't pushed through to completion.

I read with interest the kerchunk feature of xstac:

FWIW, my thinking on embedding "chunk manifests" (to use virtual zarr's term) in the STAC metadata has shifted a bit since implementing that in xstac. The chunk manifests can get quite large, which may not be the best user experience.

If I had to guess, I think it will be more common to store just the metadata needed to build an xarray Dataset from a STAC item or items (things like the variables, dims, attrs, encoding, shape, chunksize), and you'd include a STAC Asset linking to the chunk manifests. You'll hopefully be able to delay reading the Chunk Manifests, until you actually need to read data (on your Dask worker, for example; you'd read the chunk manifest, which will tell you what additional reads you need to make for the actual data).

rsignell commented 1 month ago

@TomAugspurger , okay, thanks, so basically like having STAC assets pointing to a kerchunk reference json or set of Parquet files, right?

TomAugspurger commented 1 month ago

Yep, that's the idea.