Open-EO / openeo-geopyspark-driver

OpenEO driver for GeoPySpark (Geotrellis)
Apache License 2.0
26 stars 4 forks source link

Masking in callback vs in mask not the same outcome #184

Closed Bart92 closed 2 years ago

Bart92 commented 2 years ago

We noticed when following up an issue on the forum that masking using "mask" and masking in a callback does not give the same output. Example code:

import openeo from openeo.processes import arrayelement, if, gt, divide

connection = openeo.connect("openeo-dev.vito.be").authenticate_oidc()

start_date = '2022-01-01' end_date = '2022-01-31' bands = ['B02', 'B03', 'B04', 'B05', 'B06', 'B07', 'B08', 'B8A', 'B09', 'B11', 'B12', 'CLP', 'SCL' , 'sunAzimuthAngles', 'sunZenithAngles'] spatial_extent = {'west': -74.06810760, 'east': -73.90597343, 'south': 4.689864510, 'north': 4.724080996, 'crs': 'epsg:4326'} #colombia

s2_cube = connection.load_collection( 'SENTINEL2_L2A_SENTINELHUB', spatial_extent = spatial_extent, temporal_extent = [start_date, end_date], bands = bands)

METHOD 1: USING CALLBACK

def _callback(x): clp = x.arrayelement(11) return if((clp.divide(255)).gt(0.3), x) s2_cube_a = s2_cube.apply_dimension(dimension="bands", process=_callback) s2_cube_a.download("s2_masked_metcallback.nc")

METHOD 2: USING MASK

clp = s2_cube.band("CLP") mask_clp = (clp / 255) > 0.3 s2_cube_clp = s2_cube.mask(mask_clp) s2_cube_clp.download("s2_masked_zondercallback.nc")

soxofaan commented 2 years ago

What is the difference in outcome? A large fraction of pixels different or just a couple?

Bart92 commented 2 years ago

They are completely different. Result with callback: metcallback

Result with mask: metmask

jdries commented 2 years ago

I tried it, the confusion comes from the fact that the condition needs to be reversed in the callback:


    def callback(x):
        clp = x.array_element(1)
        return if_((clp/255) < 0.3, x)

    s2_cube_a = s2_cube.apply_dimension(dimension="bands", process=callback)

By doing that, I get a much more plausible output, can you confirm?

Bart92 commented 2 years ago

I tested it and you're right, I will add that solution as well on the forum