Production error - S1Tiling don't find products to process - CCN_AWS Ukraine

[agrosu1978]

The error from the attached files has been seen when processing s1 products from Ukraine aez. Please check lines:

• 25 -> cmd start
• 216 -> Exception ukraine_s1_error_ard_failed.txt

[savmickael]

it seems that S1tiling consider that the product don't intersect the tile. Can you give me the list of products with the target tile with this error please.

Not very blocking but need to be manage properly from ewoc_s1 side.

[agrosu1978]

it seems that S1tiling consider that the product don't intersect the tile. Can you give me the list of products with the target tile with this error please.

Not very blocking but need to be manage properly from ewoc_s1 side.

Sorry, I've seen this after I deleted the data from the aws database. Currently, it is re-processing. As soon as they will fail again, I will get the list for you. Suggestion for solving this:

There may be more products like this (i am talking about other potential situations) and their number may exceed the pre-processing-error threshold, thus the tile will be automatically set as pre-processing-error for CM and CT. To avoid this, in such cases: 1: ewoc_s1 should not exit brutally, just return a known pattern (like classification case) : Uploaded 0 files to bucket | placeholder 2: Thus, I will know that the product would be safe to skip, and I will not set its status to error, but to another state (like skip) This way, these products will not enter in the computation of the pre-processing-error threshold, and the tile would be classified

[savmickael]

On AWS, the product S1B_IW_GRDH_1SDV_20191112T040231_20191112T040256_018889_023A04_C65C has no productInfo.json file which is required to convert the product to SAFE format supported by S1Tiling.

We need to handle this case by raising a clear error in ewoc_dag when the safe conversion is not possible and handle this error at processor level.

[savmickael]

The same issue raised when we give to the processor two or more products with the same elements (except last one the unique_id). This situation appears on 36TYS with products S1B_IW_GRDH_1SDV_20171101T153534_20171101T153600_008090_00E4B2_A2A5 and S1B_IW_GRDH_1SDV_20171101T153534_20171101T153600_008090_00E4B2_41B4 set in the workplan for 2018 on aez 22096.

The workplan have an issue because it put two products with similar information (NRT and Fast ?)

With these products ids, one failed and the other is downloaded but due to its siez the failed one is used by S1 tiling which indicate no products to process and the ewoc_s1 processor failed later.

[savmickael]

To solve these issues:

• an issue on the ewoc_workplan to handle double : https://github.com/WorldCereal/ewoc_work_plan/issues/120
• an issue on the ewoc_dag to handle missing json file and manage current bugged workplan: https://github.com/WorldCereal/ewoc_dataship/issues/113 and https://github.com/WorldCereal/ewoc_dataship/issues/114
• an issue on ewoc_s1 to handle the new ewoc_dag version and return clear error which will handle by the processing system: https://github.com/WorldCereal/ewoc_s1/issues/49