search

Open-EO / openeo-opensearch-client

Simple opensearch client for openeo.
Apache License 2.0
0 stars 0 forks source link

Creo catalog: filter bad product in client #9

Closed jdries closed 1 year ago

jdries commented 1 year ago

Piece of test code:


    val openSearch = CreodiasClient()

    val features = openSearch.getProducts(
      collectionId = "Sentinel2",
      (LocalDate.of(2018, 8, 12), LocalDate.of(2018, 8, 13)),
      ProjectedExtent(Extent(-3.7937789378418247,38.486414764328515,-3.5314443712734733,38.69684729114566), LatLng),
      Map[String, Any]("eo:cloud_cover" -> 90.0, "productType" -> "L2A"), correlationId = "hello", "S2MSI2A"
    )

This call returns 3 products that appear the same. The good one is: /eodata/Sentinel-2/MSI/L2A/2018/08/12/S2A_MSIL2A_20180812T105621_N0213_R094_T30SVH_20201026T165913.SAFE

I didn't check why deduplication does not remove the others, but we should get rid of them as it makes the system crash.

EmileSonneveld commented 1 year ago

S2A_MSIL2A_20180812T105621_N0208_R094_T30SVH_20180812T141312.SAFE has resolution '0' and is while S2A_MSIL2A_20180812T105621_N0213_R094_T30SVH_20201026T165913.SAFE has resolution '60'. Making them different. I think it is safe to ignore resolution when it is 0.

The published fields have 2years between them while, at the moment, only 30seconds is allowed to be considered for dedup. I'll see if it is safe to remove that check too.

S2A_MSIL2A_20180812T105621_N9999_R094_T30SVH_20220926T193221 has a different extent so should be ok. Resolution is '0' ere too.

jdries commented 1 year ago

yes, published difference can be quite big, I would go ahead as you propose

EmileSonneveld commented 1 year ago

The 0 resolution duplicate should be filtered away now. The build is uploaded to libs-release-local. Not sure if it is deployed yet. I launched integrationtests just in case