search

Element84 / earth-search

Earth Search information and issue tracking
https://earth-search.aws.element84.com/v1
27 stars 2 forks source link

Missing assets of certain products in the `sentinel-2-c1-l2a` collection #36

Open Konkrad opened 5 months ago

Konkrad commented 5 months ago

I encountered some Product in the sentinel-2-c1-l2a collection which miss assets.

Examples of products are:

The entry in the stac catalog is correct in the sense that it matches the assets available in the bucket itself:

aws s3 ls --no-sign-request s3://e84-earth-search-sentinel-data/sentinel-2-c1-l2a/34/T/DK/2018/6/S2A_T34TDK_20180603T092029_L2A/

When checking the assets in the copernicus Browser, all assets can be listed. Therefore I assume there is somewhere an error as the assets are not there.

philvarner commented 4 months ago

This does seem to be a problem.

For S2A_T34TDK_20180603T092029_L2A:

$ aws s3 ls s3://e84-earth-search-sentinel-data/sentinel-2-c1-l2a/34/T/DK/2018/6/S2A_T34TDK_20180603T092029_L2A/
2024-03-15 22:23:37     342780 AOT.tif
2024-03-15 22:23:37    2059284 B05.tif
2024-03-15 22:23:37    2141780 B06.tif
2024-03-15 22:23:37    2164206 B07.tif
2024-03-15 22:23:37    2091443 B11.tif
2024-03-15 22:23:37    2072997 B12.tif
2024-03-15 22:23:38    2166538 B8A.tif
2024-03-15 22:23:38     820996 CLD_20m.tif
2024-03-15 22:23:38      17580 L2A_PVI.jpg
2024-03-15 22:23:38     103029 L2A_PVI.tif
2024-03-15 22:23:43      13054 S2A_T34TDK_20180603T092029_L2A.json
2024-03-15 22:23:38     122674 SCL.tif
2024-03-15 22:23:38      59516 SNW_20m.tif
2024-03-15 22:23:37    1665914 WVP.tif
2024-03-15 22:23:38     194858 metadata.xml
2024-03-15 22:23:38      55291 product_metadata.xml
2024-03-15 22:23:38       1616 tileInfo.json

Sentinel-2 RODA:

$ aws s3 ls s3://sentinel-s2-l2a/tiles/34/T/DK/2018/6/3/3/ --recursive --request
-payer requester
2024-03-15 22:22:39    1510055 tiles/34/T/DK/2018/6/3/3/R10m/AOT.jp2
2024-03-15 22:22:39   35670825 tiles/34/T/DK/2018/6/3/3/R10m/B02.jp2
2024-03-15 22:22:39   36969239 tiles/34/T/DK/2018/6/3/3/R10m/B03.jp2
2024-03-15 22:22:39   36611505 tiles/34/T/DK/2018/6/3/3/R10m/B04.jp2
2024-03-15 22:22:39   43456842 tiles/34/T/DK/2018/6/3/3/R10m/B08.jp2
2024-03-15 22:22:39   55093686 tiles/34/T/DK/2018/6/3/3/R10m/TCI.jp2
2024-03-15 22:22:39   26648339 tiles/34/T/DK/2018/6/3/3/R10m/WVP.jp2
2024-03-15 22:22:39    1145553 tiles/34/T/DK/2018/6/3/3/R20m/AOT.jp2
2024-03-15 22:22:39    6610677 tiles/34/T/DK/2018/6/3/3/R20m/B01.jp2
2024-03-15 22:22:39   10373706 tiles/34/T/DK/2018/6/3/3/R20m/B02.jp2
2024-03-15 22:22:39   10739300 tiles/34/T/DK/2018/6/3/3/R20m/B03.jp2
2024-03-15 22:22:39   10834492 tiles/34/T/DK/2018/6/3/3/R20m/B04.jp2
2024-03-15 22:22:39   10753257 tiles/34/T/DK/2018/6/3/3/R20m/B05.jp2
2024-03-15 22:22:39   11844870 tiles/34/T/DK/2018/6/3/3/R20m/B06.jp2
2024-03-15 22:22:39   12131733 tiles/34/T/DK/2018/6/3/3/R20m/B07.jp2
2024-03-15 22:22:39   10751131 tiles/34/T/DK/2018/6/3/3/R20m/B11.jp2
2024-03-15 22:22:39   10540329 tiles/34/T/DK/2018/6/3/3/R20m/B12.jp2
2024-03-15 22:22:39   12085327 tiles/34/T/DK/2018/6/3/3/R20m/B8A.jp2
2024-03-15 22:22:39     686181 tiles/34/T/DK/2018/6/3/3/R20m/SCL.jp2
2024-03-15 22:22:39   14951907 tiles/34/T/DK/2018/6/3/3/R20m/TCI.jp2
2024-03-15 22:22:39    8456721 tiles/34/T/DK/2018/6/3/3/R20m/WVP.jp2
2024-03-15 22:22:39     312538 tiles/34/T/DK/2018/6/3/3/R60m/AOT.jp2
2024-03-15 22:22:39    1291087 tiles/34/T/DK/2018/6/3/3/R60m/B01.jp2
2024-03-15 22:22:39    1425283 tiles/34/T/DK/2018/6/3/3/R60m/B02.jp2
2024-03-15 22:22:39    1456062 tiles/34/T/DK/2018/6/3/3/R60m/B03.jp2
2024-03-15 22:22:39    1484523 tiles/34/T/DK/2018/6/3/3/R60m/B04.jp2
2024-03-15 22:22:39    1495557 tiles/34/T/DK/2018/6/3/3/R60m/B05.jp2
2024-03-15 22:22:39    1619823 tiles/34/T/DK/2018/6/3/3/R60m/B06.jp2
2024-03-15 22:22:39    1652170 tiles/34/T/DK/2018/6/3/3/R60m/B07.jp2
2024-03-15 22:22:39    1619157 tiles/34/T/DK/2018/6/3/3/R60m/B09.jp2
2024-03-15 22:22:39    1545233 tiles/34/T/DK/2018/6/3/3/R60m/B11.jp2
2024-03-15 22:22:39    1518592 tiles/34/T/DK/2018/6/3/3/R60m/B12.jp2
2024-03-15 22:22:39    1654884 tiles/34/T/DK/2018/6/3/3/R60m/B8A.jp2
2024-03-15 22:22:39     163938 tiles/34/T/DK/2018/6/3/3/R60m/SCL.jp2
2024-03-15 22:22:39    1893950 tiles/34/T/DK/2018/6/3/3/R60m/TCI.jp2
2024-03-15 22:22:39    1233912 tiles/34/T/DK/2018/6/3/3/R60m/WVP.jp2
2024-03-15 22:22:39       2970 tiles/34/T/DK/2018/6/3/3/auxiliary/AUX_CAMSRE
2024-03-15 22:22:39       1620 tiles/34/T/DK/2018/6/3/3/auxiliary/AUX_ECMWFT
2024-03-15 22:22:39     194858 tiles/34/T/DK/2018/6/3/3/metadata.xml
2024-03-15 22:22:40       1031 tiles/34/T/DK/2018/6/3/3/productInfo.json
2024-03-15 22:22:39      18744 tiles/34/T/DK/2018/6/3/3/qi/CLASSI_B00.jp2
2024-03-15 22:22:39     838696 tiles/34/T/DK/2018/6/3/3/qi/CLD_20m.jp2
2024-03-15 22:22:39     196311 tiles/34/T/DK/2018/6/3/3/qi/CLD_60m.jp2
2024-03-15 22:22:39      10750 tiles/34/T/DK/2018/6/3/3/qi/DETFOO_B01.jp2
2024-03-15 22:22:39      42094 tiles/34/T/DK/2018/6/3/3/qi/DETFOO_B02.jp2
2024-03-15 22:22:39      42221 tiles/34/T/DK/2018/6/3/3/qi/DETFOO_B03.jp2
2024-03-15 22:22:39      42158 tiles/34/T/DK/2018/6/3/3/qi/DETFOO_B04.jp2
2024-03-15 22:22:39      16528 tiles/34/T/DK/2018/6/3/3/qi/DETFOO_B05.jp2
2024-03-15 22:22:39      16543 tiles/34/T/DK/2018/6/3/3/qi/DETFOO_B06.jp2
2024-03-15 22:22:39      16531 tiles/34/T/DK/2018/6/3/3/qi/DETFOO_B07.jp2
2024-03-15 22:22:39      42007 tiles/34/T/DK/2018/6/3/3/qi/DETFOO_B08.jp2
2024-03-15 22:22:39      10603 tiles/34/T/DK/2018/6/3/3/qi/DETFOO_B09.jp2
2024-03-15 22:22:39      10535 tiles/34/T/DK/2018/6/3/3/qi/DETFOO_B10.jp2
2024-03-15 22:22:39      16618 tiles/34/T/DK/2018/6/3/3/qi/DETFOO_B11.jp2
2024-03-15 22:22:39      16635 tiles/34/T/DK/2018/6/3/3/qi/DETFOO_B12.jp2
2024-03-15 22:22:39      16657 tiles/34/T/DK/2018/6/3/3/qi/DETFOO_B8A.jp2
2024-03-15 22:22:39       7979 tiles/34/T/DK/2018/6/3/3/qi/FORMAT_CORRECTNESS.xml
2024-03-15 22:22:39       4065 tiles/34/T/DK/2018/6/3/3/qi/GENERAL_QUALITY.xml
2024-03-15 22:22:39       4917 tiles/34/T/DK/2018/6/3/3/qi/GEOMETRIC_QUALITY.xml
2024-03-15 22:22:39     167241 tiles/34/T/DK/2018/6/3/3/qi/L2A_PVI.jp2
2024-03-15 22:22:39       7993 tiles/34/T/DK/2018/6/3/3/qi/L2A_QUALITY.xml
2024-03-15 22:22:39      14773 tiles/34/T/DK/2018/6/3/3/qi/QUALIT_B01.jp2
2024-03-15 22:22:39      57051 tiles/34/T/DK/2018/6/3/3/qi/QUALIT_B02.jp2
2024-03-15 22:22:39      57169 tiles/34/T/DK/2018/6/3/3/qi/QUALIT_B03.jp2
2024-03-15 22:22:39      57779 tiles/34/T/DK/2018/6/3/3/qi/QUALIT_B04.jp2
2024-03-15 22:22:39      15023 tiles/34/T/DK/2018/6/3/3/qi/QUALIT_B05.jp2
2024-03-15 22:22:39      15126 tiles/34/T/DK/2018/6/3/3/qi/QUALIT_B06.jp2
2024-03-15 22:22:39      14921 tiles/34/T/DK/2018/6/3/3/qi/QUALIT_B07.jp2
2024-03-15 22:22:39      65424 tiles/34/T/DK/2018/6/3/3/qi/QUALIT_B08.jp2
2024-03-15 22:22:39      14764 tiles/34/T/DK/2018/6/3/3/qi/QUALIT_B09.jp2
2024-03-15 22:22:39      19466 tiles/34/T/DK/2018/6/3/3/qi/QUALIT_B10.jp2
2024-03-15 22:22:39      24945 tiles/34/T/DK/2018/6/3/3/qi/QUALIT_B11.jp2
2024-03-15 22:22:39      24914 tiles/34/T/DK/2018/6/3/3/qi/QUALIT_B12.jp2
2024-03-15 22:22:39      15748 tiles/34/T/DK/2018/6/3/3/qi/QUALIT_B8A.jp2
2024-03-15 22:22:39       3293 tiles/34/T/DK/2018/6/3/3/qi/SENSOR_QUALITY.xml
2024-03-15 22:22:39      32770 tiles/34/T/DK/2018/6/3/3/qi/SNW_20m.jp2
2024-03-15 22:22:39      16066 tiles/34/T/DK/2018/6/3/3/qi/SNW_60m.jp2
2024-03-15 22:22:39       1616 tiles/34/T/DK/2018/6/3/3/tileInfo.json
keithpeck commented 1 month ago

We just received S2A_T16QDK_20240704T161931_L2A via the SNS topic, which only has assets: coastal, nir09, preview, granule_metadata, tileinfo_metadata, product_metadata, thumbnail