search

microsoft / PlanetaryComputer

Issues, discussions, and information about the Microsoft Planetary Computer
https://planetarycomputer.microsoft.com/
MIT License
182 stars 7 forks source link

Sentinel-2 L2A data artefacts #291

Open dzanaga opened 11 months ago

dzanaga commented 11 months ago

Hi,

we noticed a recurring artifact in some of the L2A products. See for example: Product id: S2B_MSIL2A_20200206T153619_R068T17NQA20201001T034830 Processing Baseline: 02.12 Band B02 scaled 0 - 1000 image

vs same product (actually not the same, but this is the original L2A product from ESA) on AWS:

Product id: S2B_MSIL2A_20200206T153619_N0214_R068T17NQA20200206T194512 Processing Baseline: 02.14 Band B02 scaled 0 - 1000 image

From the products ids we can see that the product on Planetary Computer was generated on 2020-10-01 with an older version of Sen2Cor, compared to the product produced by ESA which was generated 4 hrs after the acquisition time and using a slightly more recent version of Sen2Cor. I am not sure if the artefact derives from the older Sen2Cor version or some other step in the conversion to COG...

We were wondering if you were aware of this issue (browsing around you will find more products with it), and also since ESA is currently re-processing the full archive with the more recent processing baseline, which includes several improvements, is there any plan to re-ingest the updated products (full archive), or at least replace the products showing these artefacts?

Thanks!

TomAugspurger commented 11 months ago

Thanks for opening this up.

I don't know enough about the internals of sen2cor to say whether it being from an older version is likely to cause the artifacts you're seeing, but it's possible. That or the DEM we use (https://github.com/microsoft/PlanetaryComputer/discussions/149) are the most likely culprits.

is there any plan to re-ingest the updated products (full archive), or at least replace the products showing these artefacts?

No concrete plans right now, but perhaps someday.

We most likely wouldn't do this for a subset of scenes. We want to provide an archive that's internally consistent. It would probably need to be all or nothing.

dzanaga commented 11 months ago

Thanks Tom, I wasn't aware you used a different DEM, that might be it indeed. You could confirm it by checking the PlanetDEM over that area. I have seen the artefact over Ecuador / Colombia. Hopefully it's limited to that region.

We most likely wouldn't do this for a subset of scenes. We want to provide an archive that's internally consistent. It would probably need to be all or nothing.

Ok, understood, hopefully there will be an update of the catalog once ESA is done with processing with the new baseline so that Planetary Computer data is consistent with the other providers. I'll come back when the back-processing is complete :)

dzanaga commented 11 months ago

@TomAugspurger The back-processing should complete by the end of the year. They are now finishing 2020. It will be possible to ingest the new collection in the new year.

purkristin commented 4 weeks ago

Are there any advancements on this topic? I can still find datasets showing similar striping. image

E.g. here a snippet from tile 18NTG 2016-01-23

dzanaga commented 3 weeks ago

Meanwhile backprocessing of Sentinel-2 collection 1 on the Copernicus Data Space Ecosystem is completed up to 2021. More info and updates here: https://dataspace.copernicus.eu/news/2024-9-2-sentinel-2-collection-1-products-availability