[BUG] [Sentinel-2] Sentinel-2 Level-2 RS add-on fails.

pcuq-ads commented 1 year ago

Environment:

Platform: OPS Orange Cloud
Configuration: OPS
Infrastructure v1.5.0
RS core ingestion, distribution, data lifecycle, compression 1.12.0-rc1
RS core metadata 1.12.1-rc1 with workaround
RS core Monitoring 1.5.0-rc1
RS add-on S2-L0u 1.6.0-rc1
RS add-on S2-L0c 1.6.0-rc1
RS add-on S1-L1 1.6.0-rc1
RS add-on S1-L2 1.6.0-rc1

Traceability:

Test(s): TST_PRO_WF_S2L2
Requirements:

Current Behavior: Sentinel-2 Level-2 processing fails.

Expected Behavior: All RS-add-on processing shall be up and running.

Steps To Reproduce: Obvious. Deploy and start the S2-L2 chain and check production.

Test execution artefacts (i.e. logs, screenshots…)

Whenever possible, first analysis of the root cause Information from @nleconte-csgroup :

"Currently, level 2 of Sentinel 2 in OPS is not working. But it is easily bypassed. On Sentinel 2, the versions of IPFs to use for processing are defined in an AUX (another S2 peculiarity):

GIP_PROBAS for level 1
GIP_PROBA2 for level 2

These versions can be bypassed via a configuraiton file in the Python orchestrator that comes from the Production Service. This is practical, as it avoids having an AUX that tells us to use version 6.1.4 of the IPF when we are running version 6.1.0, for example.

In this AUX, there is also the "baseline version". This is the number at the end of the products, for example 04.00 for this DS: S2B_OPER_MSI_L1C_DS_REFS_20230329T135633_S20230325T203106_N04.00

Unfortunately, there is a processor at level 2 (Sen2Cor) that will still read the baseline version in this AUX no matter what (another S2 peculiarity).

Currently, the PROBA2 AUX in OPS gives the baseline version as 05.09. And so at level L2, this IPF (Sen2Cor) starts to generate an L2A DS with this name: S2B_OPER_MSI_L2A_DS_REFS_20230404T095354S20230325T203106N05.09, but the previous steps have generated an L2A DS S2B_OPER_MSI_L2A_DS_REFS_20230404T095354S20230325T203106N04.00.

After discussion with our S2 expert and Cyrille, the easiest solution is to disable the ingestion of new PROBA2 (via regex at the AUXIP ingestion level), delete the PROBA2 that mention the 05.09 baseline from our catalog, and restart the S2 L2 processing."

Bug Generic Definition of Ready (DoR)

[X] The affect version in which the bug has been found is mentioned
[X] The context and environment of the bug is detailed
[X] The description of the bug is clear and unambiguous
[X] The procedure (steps) to reproduce the bug is clearly detailed
[ ] The tested User Story / features is linked to the bug if available
[ ] Logs are attached if available
[ ] A data set attached if available

Bug Generic Definition of Done (DoD)

[ ] the modification implemented (the solution to fix the bug) is described in the bug.
[ ] Unit tests & Continuous integration performed - Test results available - Structural Test coverage reported by SONAR
[ ] Code committed in GIT with right tag or Analysis/Trade Off documentation up-to-date in reference-system-documentation repository
[ ] Code is compliant with coding rules (SONAR Report as evidence)
[ ] Acceptance criteria of the related User story are checked and Passed

pcuq-ads commented 1 year ago

Dear @nleconte-csgroup ,

I have accepted your pull request : https://github.com/COPRS/rs-config/pull/123 Please proceed with the change and deploy the fix in operation. Please, update also the "PRE Integration - tests" document to share S2 L2 processing behaviour.

Regards