Open-EO / openeo-geotrellis-extensions

Java/Scala extensions for Geotrellis, for use with OpenEO GeoPySpark backend.
Apache License 2.0
5 stars 3 forks source link

fix TERRASCOPE_S1_SLC_COHERENCE_V1 collection test #297

Closed bossie closed 1 month ago

bossie commented 1 month ago

Terrascope collection test openeo-collection-tests.test_layers.test_layer[TERRASCOPE_S1_SLC_COHERENCE_V1-2020-03-01-2020-04-01] has been failing for some time:

AssertionError: DataFrame.iloc[:, 88] (column name="(44, 0)") are different

DataFrame.iloc[:, 88] (column name="(44, 0)") values are different (10.0 %)
[index]: [2020-03-01T00:00:00Z, 2020-03-02T00:00:00Z, 2020-03-03T00:00:00Z, 2020-03-04T00:00:00Z, 2020-03-05T00:00:00Z, 2020-03-06T00:00:00Z, 2020-03-07T00:00:00Z, 2020-03-08T00:00:00Z, 2020-03-09T00:00:00Z, 2020-03-10T00:00:00Z, 2020-03-11T00:00:00Z, 2020-03-12T00:00:00Z, 2020-03-13T00:00:00Z, 2020-03-14T00:00:00Z, 2020-03-15T00:00:00Z, 2020-03-16T00:00:00Z, 2020-03-17T00:00:00Z, 2020-03-18T00:00:00Z, 2020-03-19T00:00:00Z, 2020-03-20T00:00:00Z, 2020-03-21T00:00:00Z, 2020-03-22T00:00:00Z, 2020-03-23T00:00:00Z, 2020-03-24T00:00:00Z, 2020-03-25T00:00:00Z, 2020-03-27T00:00:00Z, 2020-03-28T00:00:00Z, 2020-03-29T00:00:00Z, 2020-03-30T00:00:00Z, 2020-03-31T00:00:00Z]
[left]:  [nan, nan, nan, nan, nan, 73.6774193548387, nan, nan, 63.32258064516129, nan, nan, 84.61290322580645, nan, nan, nan, nan, nan, 79.16129032258064, nan, nan, 71.48387096774194, nan, nan, 81.0, nan, nan, nan, nan, 117.80645161290325, nan]
[right]: [nan, nan, 75.35483870967742, nan, nan, 73.6774193548387, nan, nan, 63.32258064516129, nan, nan, 84.61290322580645, nan, nan, 98.35483870967742, nan, nan, 79.16129032258064, nan, nan, 71.48387096774194, nan, nan, 81.0, nan, 97.09677419354838, nan, nan, 117.80645161290325, nan]
At positional index 2, first diff: nan != 75.35483870967742

It's not a matter of changed input data because running this test against prod and dev succeeds.

bossie commented 1 month ago

I tracked down the difference in behavior to commit 6ff3ba7cc8014f88d8e01bee7e5eb6f8b26aa5ad in the context of #280.

Looks like the change is for the better* so I'm just going to update the reference data.

FYI @EmileSonneveld

EmileSonneveld commented 1 month ago

This made me reailise it needs to be added to the changelog: https://github.com/Open-EO/openeo-geopyspark-driver/commit/2fccdf688f2ed8aacbac8a108802a97a46a1c5cf

bossie commented 1 month ago

The test passes again.