Open-EO / openeo-opensearch-client

Simple opensearch client for openeo.
Apache License 2.0
0 stars 0 forks source link

add deduplication to oscars based catalog #6

Closed jdries closed 1 year ago

jdries commented 1 year ago

oscars catalog may also return multiple products, this is an example: https://services.terrascope.be/catalogue/products?collection=urn%3Aeop%3AVITO%3ATERRASCOPE_S2_TOC_V2&bbox=4.980226799009085%2C51.2479687147142%2C5.107490137869115%2C51.293636314627435&sortKeys=title&startIndex=1&accessedFrom=MEP&clientId=c-e1d1f77184aa4de68b1d7753f77f68f7_2&start=2018-08-14T00%3A00%3A00Z&end=2018-08-14T23%3A59%3A59.999999999Z

(productVersion is different)

deduplication can normally be very similar to creodias approach

EmileSonneveld commented 1 year ago

In the oscars dataset an extra duplicate was removed by this code. I could not check if the image content was a prefect duplicate, but the properties are all the same. Removing duplicated feature(s): 'urn:eop:VITO:CGS_S2_RAD_L2_V200'. Keeping the Latest published one: 'urn:eop:VITO:CGS_S2_RAD_L2'

@jdries , looks good to merge to you?

EmileSonneveld commented 1 year ago

Added the response JSON also an extra test.

EmileSonneveld commented 1 year ago

There was no need to parse productVersion. I assume that later products will always have a later published date. (As is the case here)