[BUG] [S3 SL2] Task table is incorrect when several SL1 products are available

[SYTHIER-ADS]

Environment:

• Platform: OPS Orange Cloud
• Configuration: OPS
• Infrastructure v1.5.0
• RS core ingestion, distribution, data lifecycle, compression 1.12.0-rc1
• RS core metadata 1.12.1-rc1 with workaround
• RS add-on S3-SL2-NRT 1.12.0-rc1

Traceability:

• Test(s):
  • TST_PRO_WF_S3L0
  • TST_PRO_WF_S3L1
  • TST_PRO_WF_S3L2
• Requirements: RS-REQ-SRD-004/1

Current Behavior: S3 SL2 is not able to handle when several SL1 products have been generated covering the same period (reprocessing case). In this case it add to the task table all the products.

Expected Behavior: Only one product per time coverage is expected to be added to the task table (keeping the one with the generation date closest to the generation date of the product triggering the processing).

Steps To Reproduce: Trigger SL2 with several SL1 input covering the same time span in the metadata catalog.

Test execution artefacts (i.e. logs, screenshots…) You can have the log of the execution raising the error in the ops-rs-failed-workdir bucket and using the following execution: s3-sl2-nrt-preint-part1-execution-worker-v7-76fddb6fb9-gh7m8_S3A_SL_1_RBT__20230409T190220_20230409T190720_20230418T171839_0299_097_270____LN3_D_NR_002.SEN3_9a4e23af-b558-48de-b09a-cd3de7e7ef7a_0

Whenever possible, first analysis of the root cause Hypothesis : The preparation worker is performing a request to the metadata catalog and it doesn't filter the results as expected.

Bug Generic Definition of Ready (DoR)

• [X] The affect version in which the bug has been found is mentioned
• [X] The context and environment of the bug is detailed
• [X] The description of the bug is clear and unambiguous
• [X] The procedure (steps) to reproduce the bug is clearly detailed
• [ ] The tested User Story / features is linked to the bug if available
• [ ] Logs are attached if available
• [X] A data set attached if available

Bug Generic Definition of Done (DoD)

• [ ] the modification implemented (the solution to fix the bug) is described in the bug.
• [ ] Unit tests & Continuous integration performed - Test results available - Structural Test coverage reported by SONAR
• [ ] Code committed in GIT with right tag or Analysis/Trade Off documentation up-to-date in reference-system-documentation repository
• [ ] Code is compliant with coding rules (SONAR Report as evidence)
• [ ] Acceptance criteria of the related User story are checked and Passed

[SYTHIER-ADS]

The workaround in operation is currently to delete manually the duplicated product.

[LAQU156]

IVV_CCB_2023_w17 : Moved into "Accepted Werum" for analysis, Priority major (workaround)

[LAQU156]

Werum_CCB_2023_w17 : Moved into "Product Backlog" for observation (~1 month)

[pcuq-ads]

RSRRv2_SystemCCB : to be fixed in phase 1.

[w-fsi]

As pointed out in the last CCB Werum's position is that the current behaviour is as expected. The tasktable as provided by the IPF developer specifies the following input:

                            <Input>
                                <Mode>ALWAYS</Mode>
                                <Mandatory>Yes</Mandatory>
                                <List_of_Alternatives count="1">
                                    <!-- SLSTR L1  product -->
                                    <Alternative>
                                        <Order>1</Order>
                                        <Origin>DB</Origin>
                                        <Retrieval_Mode>LatestValIntersect</Retrieval_Mode>
                                        <T0>20</T0>
                                        <T1>85</T1>
                                        <File_Type>SL_1_RBT___</File_Type>
                                        <File_Name_Type>Physical</File_Name_Type>
                                    </Alternative>
                                </List_of_Alternatives>
                            </Input>

The selection policy ValIntersect allows the selection of multiple products and thus it is expected that multiple products might be returned. We understand the issue you're having with the reprocessing scenario, but it is a generic interface and working as expected. This is the position of the team and the project management as any modification there would be basically a change to the existing system.

A suitable workaround would be to modify the selection policy the LatestValIntersect as this will always just provide a single product. We are not aware about any specifications that the first input needs to be removed and this might have more consequences as currently expected. E.g. as SR2 does list SR1 and MW1 as input and does not have technically a main input. Handling these scenarios are working fine with the current implementation if these needs to be ignored a new wait mechanism would be required to ensure that the production is ready.

This issue can either be handled to modify the selection policy or to be refused.

[LAQU156]

Werum_CCB_2023_w18 : Action @OPS : Test on SL2 addon. Be aware that this behaviour could happen to the other level 2 addons.

[LAQU156]

Werum_CCB_2023_w21 : Action on @pcuq-ads side

[LAQU156]

Werum_CCB_2023_w30 : @Woljtek please check if the workaround works

[pcuq-ads]

During the CCB, there is a need to discuss about #1043.

[pcuq-ads]

Werum_CCB_2023_w30 : The feature provided for story #1043 could be used for that chain also.

[suberti-ads]

WA has been deactivate and should be solved by #1043

[LAQU156]

System_CCB_2023_w32 : Will be fixed with #1043

[pcuq-ads]

I remove limitation because Issue 1043 is closed.