Closed rabernat closed 1 year ago
FYI the sentinel-2 items in https://earth-search.aws.element84.com/v0 are being updated and this is breaking also some of our code https://github.com/cogeotiff/rio-tiler-pds/issues/63
Thanks for the tip @vincentsarago! I determined that the problematic asset was the overview
one. By specifying specific assets, I was able to move forward - e.g.
stack = stackstac.stack(items, assets=['B01', 'B02', 'B03'])
All documentation examples fail
@rabernat would you mind updating the title of this issue? It doesn't seem like "all documentation examples" is accurate. What you're running here (stack[0, 0].compute()
) isn't in the documentation example you reference. And asset 0 that you're selecting is the overview
that has 3 bands, so it makes sense that that fails.
I can run the basic example you've linked just fine. I'm not aware of other examples that are failing; could you confirm if there are other issues?
It is very common for STAC assets (e.g. COGs) to have multiple bands, so I really don't understand this limitation
See https://github.com/gjoseph92/stackstac/issues/62. It requires a bit more infrastructure to get the chunking correct in Dask, so I didn't add it to the MVP (which stackstac still more or less is). Fortunately, cloud providers seem to be offering most datasets with each analytic band in a separate COG, so often this isn't too much of an issue, as you've found.
Hi @gjoseph92 - sorry for the overly broad issue title. I have just edited it now that I understand the root cause better. My goal was to provide a highly simplified example (i.e. MRE), but I now see that the "basic example" avoids this by selecting non-problematic bands.
I also experienced the same issue on the cluster example. After having both examples fail, I incorrectly extrapolated that "all the examples were broken".
Sorry again for what probably came across as a hostile issue, when the root cause was actually my error. 🤦
FWIW, I do find it highly confusing that stackstac creates datasets that are not necessarily computable. I would much prefer to have the error raised when calling stack
, rather than at compute time. While the examples are not, in fact, broken, making minor modifications (as I did) does lead to errors.
Is the number of bands in each asset part of the STAC metadata? Or can it only be discovered by opening the file.
Is the number of bands in each asset part of the STAC metadata?
No, at least not universally. That's why, unfortunately, stackstac creates datasets that aren't computable. There's no required metadata in STAC about the number of bands in an asset, so there will always be cases where we can't catch the error until runtime.
There are some optional STAC extensions that can provide the necessary metadata. stackstac does not currently use these to select assets (or exclude assets that won't work), but it could. Both eo:bands
and raster
can indicate the number of bands in an asset. raster
didn't exist when I wrote stackstac, but I'd like to start using it for a few reasons: https://github.com/gjoseph92/stackstac/issues/91 https://github.com/gjoseph92/stackstac/issues/63 https://github.com/gjoseph92/stackstac/issues/62#issuecomment-868090011.
The tricky part is that STAC also lets you specify item_assets
at the collection level, and stackstac is designed to just take a sequence of Items, not a collection, so for providers that use that approach, we'd be missing the metadata. (earth-search.aws.element84.com
is one example of this.)
So adding some logic to pre-filter assets with >1 band according to eo:bands
or raster
would be pretty straightforward if those are defined at the asset level, but require a bit more refactoring to handle the item_assets
case.
Thanks for explaining Gabe, and sorry for the noise on the issue tracker.
I tried stackstac for the first time today. I started by just running the Basic example from the documentation. Here is the code I ran
So far so good. Then I tried to actually compute something
This gave the following error
The relevant code is quite old:
https://github.com/gjoseph92/stackstac/blob/d53f3dc94e49b95b950201f9e53539a92b1458b6/stackstac/rio_reader.py#L338-L344
so this doesn't seem like a recent change. It is very common for STAC assets (e.g. COGs) to have multiple bands, so I really don't understand this limitation. All the other examples I tried fail in the same way.
What am I missing here?
I ran this on https://us-central1-b.gcp.pangeo.io/ with the following packages: