crash for specific areas and year minicubes

[cmosig]

code to reproduce:

da = sentle.process(
            target_crs=equi7_crs,
            bound_left=bound_left,
            bound_bottom=bound_bottom,
            bound_right=bound_right,
            bound_top=bound_top,
            datetime=f"{year}-01-01/{year}-12-31",
            target_resolution=res,
            dask_scheduler_port=10012,
            dask_dashboard_address='127.0.0.1:37382',
            S2_mask_snow=True,
            S2_cloud_classification=True,
            S2_cloud_classification_device="cuda",
            S1_assets=["vv", "vh"],
            S2_apply_snow_mask=True,
            S2_apply_cloud_mask=True,
            time_composite_freq="7d",
            # NOTE clemens: this can be set to 40
            num_workers=40,
        )

cubes:

minicube 114,115, 125,126 = 2022
minicube 242,479 = 2015 - 2024
minicube 856 = 2017, 2020, 2023

error message

2024-10-01 10:47:46,672 - INFO - > Save the Minicube 856 ... 
2024-10-01 10:48:09,062 - distributed.worker - ERROR - Compute Failed 
Key: ('process_ptile-this-store-map-a59daf6a505f970edd2ffbabbfb76020', 51, 0, 0, 0) 
State: executing 
Function: execute_task 
args: ((<function store_chunk at 0x7b2ee39c07c0>, (<built-in function getitem>, (<function map_blocks.<locals>._wrapper at 0x7b2e1a49a700>, <function process_ptile at 0x7b2e5cfe4180>, [(<class 'xarray.core.dataset.Dataset'>, (<class 'dict'>, [['full_like-e1c196bad4f2a81b5adbd1369d67f95e', (('time', 'band', 'y', 'x'), 
array([[[[nan, nan, nan, ..., nan, nan, nan],], [nan ..........................] 
kwargs: {} 
Exception: 'AttributeError("\'NoneType\' object has no attribute \'index\'")' 
Traceback: ' File "/net/home/fmueller/.conda/envs/emp/lib/python3.12/site-packages/xarray/core/parallel.py", line 351, in _wrapper\n result = func(*converted_args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/net/home/fmueller/.conda/envs/emp/lib/python3.12/site-packages/sentle/sentle.py", line 391, in process_ptile\n return process_ptile_S2_dispatcher(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/net/home/fmueller/.conda/envs/emp/lib/python3.12/site-packages/sentle/sentle.py", line 672, in process_ptile_S2_dispatcher\n ptile_timestamp, ptile_array_bands = process_ptile_S2(\n ^^^^^^^^^^^^^^^^^\n File "/net/home/fmueller/.conda/envs/emp/lib/python3.12/site-packages/sentle/sentle.py", line 840, in process_ptile_S2\n subtile_array_bands.index(band) for band in S2_cloud_prob_bands\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n'

2024-10-01 10:48:09,079 - ERROR - An error occurred for index 856: 'NoneType' object has no attribute 'index'

[cmosig]

line of code where it crashes: https://github.com/cmosig/sentle/blob/10432095d3f06f81ae1f83c910f78e70010d9176/sentle/sentle.py#L840

[Franziska279]

error happens with the following mini-cubes (CRS = EPSG:27705):

Minicube: 114, bound_left = 8698880, bound_bottom = 2652160, bound_right = 8704000, bound_top = 2657280 Minicube: 115, bound_left = 8698880, bound_bottom = 2657280, bound_right = 8704000, bound_top = 2662400 Minicube: 125, bound_left = 8709120, bound_bottom = 2626560, bound_right = 8714240, bound_top = 2631680 Minicube: 126, bound_left = 8709120, bound_bottom = 2641920, bound_right = 8714240, bound_top = 2647040 Minicube: 242, bound_left = 8760320, bound_bottom = 2621440, bound_right = 8765440, bound_top = 2626560 Minicube: 479, bound_left = 9011200, bound_bottom = 2723840, bound_right = 9016320, bound_top = 2728960 Minicube: 856, bound_left = 9594880, bound_bottom = 3092480, bound_right = 9600000, bound_top = 3097600

[cmosig]

Testing minicube 114.

First observation, here we have one of these locations on earth where the stac interface returns an invalid href. E.g: https://sentinel2l2a01.blob.core.windows.net/sentinel2-l2/15/S/VR/2022/06/22/S2B_MSIL2A_20220622T164849_N0400_R026_T15SVR_20220623T061111.SAFE/GRANULE/L2A_T15SVR_A027651_20220622T170254/IMG_DATA/R20m/T15SVR_20220622T164849_B11_20m.tif?st=2024-10-19T16%3A54%3A31Z&se=2024-10-21T16%3A54%3A31Z&sp=rl&sv=2024-05-04&sr=c&skoid=9c8ff44a-6a2c-4dfb-b298-1c9212f64d9a&sktid=72f988bf-86f1-41af-91ab-2d7cd011db47&skt=2024-10-20T02%3A45%3A02Z&ske=2024-10-27T02%3A45%3A02Z&sks=b&skv=2024-05-04&sig=wAafDdXCEHOfGCyn%2BrKz07x%2B7a7MrWZEK%2B5tXF7r53s%3D

[cmosig]

Resolved with above commit, but somehow sentinel 1 looks for some cases weird. Not even sure if related:

[cmosig]

seems to be fixed with: a53e776cb207420fa5a122c11f699d3095017a4e