Element84 / earth-search

Earth Search information and issue tracking
https://earth-search.aws.element84.com/v1
27 stars 2 forks source link

L2A: wvp and aot assets have incorrect resolution and shape info #8

Closed ircwaves closed 8 months ago

ircwaves commented 1 year ago

TL;DR

The L2A STAC Item wvp and aod assets seem to be populated with the wrong resolution (and correspondingly incorrect array shape):

wvp: gdal:10 stac:20
aot: gdal:60 stac:20

Full check

I checked all the assets. Here's the output:

{
  "id": "S2B_15TUG_20230713_0_L2A",
  "collection": "sentinel-2-l2a",
  "updated": "2023-07-14T03:41:37.283Z",
  "created": "2023-07-14T03:41:37.283Z"
}
aot: gdal:60 stac:20
blue: gdal:10 stac:10
coastal: gdal:60 stac:60
green: gdal:10 stac:10
nir: gdal:10 stac:10
nir08: gdal:20 stac:20
nir09: gdal:60 stac:60
red: gdal:10 stac:10
rededge1: gdal:20 stac:20
rededge2: gdal:20 stac:20
rededge3: gdal:20 stac:20
scl: gdal:20 stac:20
swir16: gdal:20 stac:20
swir22: gdal:20 stac:20
wvp: gdal:10 stac:20

Here's the script that did the checking:

#!/bin/bash

{
    which stac-client &&
    which jq &&
    which gdalinfo &&
    which sed &&
    which grep;
} >/dev/null || { echo "could not find one or more required utilities" ; exit -1 ; }

STAC_ITEM=$(stac-client search \
                        -c sentinel-2-l2a \
                        --max-items 1 \
                        --datetime 2023-07-13 \
                        --bbox -94.8249798 41.9412431 -94.8201349 41.9443611 \
                        https://earth-search.aws.element84.com/v1 \
                | jq -rc '.features[0]')
echo $STAC_ITEM | jq '''{
  "id": .id,
  "collection": .collection,
  "updated": .properties.updated,
   "created": .properties.created
}
'''

for ASSET in $(echo $STAC_ITEM|jq -r '.assets|keys[]') ;
do
    # skip jp2 and metadata assets
    if [ "${ASSET#*-}" = "jp2" \
                       -o "${ASSET}" = "thumbnail" \
                       -o "$ASSET" = "preview" \
                       -o "$ASSET" = "visual" \
                       -o "${ASSET#*_}" = "metadata" \
       ] ; then
        continue ;
    fi
    echo -n "$ASSET: ";  # start the string to assoc any errors with this asset
    AOBJ=$(echo $STAC_ITEM | jq -r '.assets."'${ASSET}'"') ;
    HREF="$(echo $AOBJ | jq -r '.href')" ;
    FROM_GDAL=$(gdalinfo /vsicurl/$HREF  \
                    | grep "Pixel Size" \
                    | sed -e 's/.*(\([^\.]*\)\..*/\1/g') ;
    FROM_STAC=$(echo $STAC_ITEM \
                    | jq -cr '.assets."'${ASSET}'"."raster:bands"[0].spatial_resolution')
    echo "gdal:${FROM_GDAL} stac:${FROM_STAC}"
done
philvarner commented 9 months ago

The underlying issue is that the v0 workflow that creates the cogs uses the AOT 60m and WVP 10m asset, but then the task that creates the v1 item uses the non-resolutioned asset names ("aot" and "wvp") that are 20m (the native resolution of these products) instead of the correct resolutioned asset names, e.g., "aot_60m" and "wvp_10m".

We could fix this, but then all of the existing ones would still be wrong, and we're moving to a new collection where this will be correct.

gadomski commented 8 months ago

This will be fixed by the reprocessing. Closing as fixed, and we should re-open if it's not actually fixed.