Element84 / earth-search

Earth Search information and issue tracking
https://earth-search.aws.element84.com/v1
27 stars 2 forks source link

not able to query imagery correctly #34

Open krishnaglodha opened 5 months ago

krishnaglodha commented 5 months ago

I'm trying to find imagery based on the geometry, my setup looks like this.

stac_endpoint = "https://earth-search.aws.element84.com/v0/search"
start = datetime.datetime.strptime("2024-03-14", "%Y-%m-%d").strftime("%Y-%m-%dT00:00:00Z")
end = datetime.datetime.strptime("2024-03-24", "%Y-%m-%d").strftime("%Y-%m-%dT23:59:59Z")

# POST body
query = {
    "collections": ["sentinel-s2-l2a"],
    "datetime": f"{start}/{end}",
    "query": {
        "eo:cloud_cover": {
            "lt": 100
        },
    },
    "intersects": geojson["features"][0]["geometry"],
    "limit": 100,
    "fields": {
      'include': ['id', 'properties.datetime', 'properties.eo:cloud_cover'],  # This will limit the size of returned body
      'exclude': ['assets', 'links']  # This will limit the size of returned body
    }
}
# POST Headers
headers = {
    "Content-Type": "application/json",
    "Accept-Encoding": "gzip",
    "Accept": "application/geo+json",
}

data = httpx.post(stac_endpoint, headers=headers, json=query).json()

I'm getting 0 results, whereas when I try same with https://planetarycomputer.microsoft.com/api/stac/v1/search it works fine, I also confirmed for the same geometry on https://apps.sentinel-hub.com/requests-builder/, I can see catalog showing dates available in same range.

What might be going wrong ?

krishnaglodha commented 5 months ago

PS: I also tried changing v0 to v1 , but no luck

m-mohr commented 5 months ago

Looks similar to https://github.com/Element84/earth-search/issues/30 Did you try slowly reducing the criteria until you get a result? e.g. start with removing the cloud cover query, then maybe replace intersects with bbox, try without fields extension, ...

matthewhanson commented 5 months ago

Hello @krishnaglodha, I'm not sure what your geometry is, but my first guess would be the same as @m-mohr, that maybe there are no results. There are some known missing data regions in the sentinel-2-l2a collection which is the older collection.

Suggest you try the sentinel-2-c1-l2a collection. Also check out the README in this repo and the links to the mailing list archive for more information about the new sentinel-2-c1-l2a collection if you haven't already.

krishnaglodha commented 5 months ago

Looks similar to #30 Did you try slowly reducing the criteria until you get a result? e.g. start with removing the cloud cover query, then maybe replace intersects with bbox, try without fields extension, ...

Hi, I tried keeping things simple, just collection , date and intersects geometry. but still no result

krishnaglodha commented 5 months ago

Hello @krishnaglodha, I'm not sure what your geometry is, but my first guess would be the same as @m-mohr, that maybe there are no results. There are some known missing data regions in the sentinel-2-l2a collection which is the older collection.

Suggest you try the sentinel-2-c1-l2a collection. Also check out the README in this repo and the links to the mailing list archive for more information about the new sentinel-2-c1-l2a collection if you haven't already.

I thought so too, ,but when I try same with https://planetarycomputer.microsoft.com/api/stac/v1/search it works fine, I also confirmed for the same geometry on https://apps.sentinel-hub.com/requests-builder/, I can see catalog showing dates available in same range.

krishnaglodha commented 5 months ago

I tried with sentinel-2-c1-l2a and it works :)

krishnaglodha commented 5 months ago

Is there any document mentioning when you use sentinel-2-c1-l2a vs sentinel-s2-l2a , which can help users to overcome this issue ?

tylere commented 5 months ago

I would expect that the metadata for a STAC Collection would fully describe what the collection contains, why it differs from other collections, and whether it has been superseded by another collection. However, in this case the metadata entries are rather terse, and probably only helpful if you already know the collection name you are looking for.

https://earth-search.aws.element84.com/v1/collections

"type": "Collection",
"id": "sentinel-2-l2a",
"stac_version": "1.0.0",
"description": "Global Sentinel-2 data from the Multispectral Instrument (MSI) onboard Sentinel-2",
"type": "Collection",
"id": "sentinel-2-l1c",
"stac_version": "1.0.0",
"description": "Global Sentinel-2 data from the Multispectral Instrument (MSI) onboard Sentinel-2",
"type": "Collection",
"id": "sentinel-2-c1-l2a",
"title": "Sentinel-2 Collection 1 Level-2A",
"description": "Sentinel-2 Collection 1 L2A, data from the Multispectral Instrument (MSI) onboard Sentinel-2",
"stac_version": "1.0.0",
fmigneault commented 5 months ago

https://earth-search.aws.element84.com/v1/collections/sentinel-2-c1-l2a/queryables vs https://earth-search.aws.element84.com/v1/collections/sentinel-s2-l2a/queryables is what to look for. The 2nd is "not found", therefore no queryable supported.

matthewhanson commented 5 months ago

@krishnaglodha Everyone should be using the new sentinel-2-c1-l2a collection unless they need data older than about 2019....ESA is still reprocessing the archive and once they are done there will be no reason to use the old collection. This has been discussed in the mailing list, and we were hoping that the backprocessing would be done by now but looks like we need to be more explicit in the documentation.

@tylere good suggestion, I'll create a ticket for us to update the collection metadata.

m-mohr commented 5 months ago

@matthewhanson Consider the version extension, deprecated and link to the next extension (as discussed in the STAC call today).

philvarner commented 4 months ago

I don't know if the original question was answered, but the issue is that the collection names are different in the different apis:

So the query of sentinel-s2-l2a against v0 for last month had zero results because the data only goes to 2022, and against v1 had zero results because sentinel-s2-l2a should be sentinel-2-l2a