[BUG] S3 PUG NTC missing SL_2_FRP products in filter, tasktable routing and pdu config

[eroan-marie]

Environment:

• Delivery tag: 1.14.0-rc1
• Platform: OPS Orange Cloud
• Configuration:

Traceability:

Current Behavior:

Can't process SL_2_FRP products

Expected Behavior:

SL_2_FRP products should be processed

Steps To Reproduce:

With default configuration try ingesting SL_2_FRP products

Test execution artefacts (i.e. logs, screenshots…) Tip: You can attach images or log files by dragging & dropping, selecting or pasting them.

Whenever possible, first analysis of the root cause

WORKAROUND Add these lines in stream-parameters.properties :

app.preparation-worker.tasktable.routing.SL_2_FRP____A=TaskTable.PUG_SL_2_FRP.03.xml

app.preparation-worker.tasktable.routing.SL_2_FRP____B=TaskTable.PUG_SL_2_FRP.03.xml

app.preparation-worker.pdu.config.SL_2_FRP___.type=FRAME
app.preparation-worker.pdu.config.SL_2_FRP___.length-in-s=180
app.preparation-worker.pdu.config.SL_2_FRP___.gap-threshhold-in-s=0.2
app.preparation-worker.pdu.config.SL_2_FRP___.dyn-proc-params.facilityName=LN3
app.preparation-worker.pdu.config.SL_2_FRP___.dyn-proc-params.hardwareName=O

app.housekeep.tasktable.routing.SL_2_FRP____A=TaskTable.PUG_SL_2_FRP.03.xml

app.housekeep.tasktable.routing.SL_2_FRP____B=TaskTable.PUG_SL_2_FRP.03.xml

app.housekeep.pdu.config.SL_2_FRP___.type=FRAME
app.housekeep.pdu.config.SL_2_FRP___.length-in-s=180
app.housekeep.pdu.config.SL_2_FRP___.gap-threshhold-in-s=0.2
app.housekeep.pdu.config.SL_2_FRP___.dyn-proc-params.facilityName=LN3
app.housekeep.pdu.config.SL_2_FRP___.dyn-proc-params.hardwareName=O

And replace filter with this :

((payload.productFamily == 'S3_AUX') && (payload.keyObjectStorage matches '^S3\\w{2}(AX___(BB2|FPO|FRO|OSF)_AX|(OL|SL)_(1|2)_(PCP|PLT)BAX|SY_2_(CVS|PCP|PLT|PVS)BAX).*')) || ((payload.productFamily == 'S3_L1_NTC') && (payload.keyObjectStorage matches '^^S3\\w{2}(OL_1_E(F|R)R|SL_1_RBT|SR_1_SRA).*')) || ((payload.productFamily == 'S3_L2_NTC') && (payload.keyObjectStorage matches '^^S3\\w{2}(OL_2_L(F|R)R|SL_2_LST|SL_2_FRP|SR_2_LAN|SY_2_(AOD|SYN|V10|VG1)).*'))

Bug Generic Definition of Ready (DoR)

• [X] The affect version in which the bug has been found is mentioned
• [X] The context and environment of the bug is detailed
• [X] The description of the bug is clear and unambiguous
• [X] The procedure (steps) to reproduce the bug is clearly detailed
• [ ] The tested User Story / features is linked to the bug if available
• [ ] Logs are attached if available
• [ ] A data set attached if available

Bug Generic Definition of Done (DoD)

• [ ] the modification implemented (the solution to fix the bug) is described in the bug.
• [ ] Unit tests & Continuous integration performed - Test results available - Structural Test coverage reported by SONAR
• [ ] Code committed in GIT with right tag or Analysis/Trade Off documentation up-to-date in reference-system-documentation repository
• [ ] Code is compliant with coding rules (SONAR Report as evidence)
• [ ] Acceptance criteria of the related User story are checked and Passed

[pcuq-ads]

IVV_CCB_2023_w29 : moved to WERUM dashboard for configuration update.

[Woljtek]

Werum_CCB_2023_w29: Accepted. Will be fixed in the next version 1.14.0-rc2.

[w-jka]

We updated the default configuration. @Woljtek @eroan-marie please note, that the same configuration was missing in the NRT configuration for the pug. We included it there as well. The Tasktables are already part of the configmap and also included in the volume mounts.

The complete list of changes for this issue: pug-nrt

app.message-filter.filter.function.expression=((payload.productFamily == 'S3_AUX') && (payload.keyObjectStorage matches '^S3\\w{2}(AX___(BB2|FPO|FRO|OSF)_AX|(OL|SL)_(1|2)_(PCP|PLT)BAX|SY_2_(CVS|PCP|PLT|PVS)BAX).*')) || ((payload.productFamily == 'S3_L0') && (payload.keyObjectStorage matches '^^S3\\w{2}(OL_0_EFR|SL_0_SLT|SR_0_SRA|TM_0_NAT).*')) || ((payload.productFamily == 'S3_L1_NRT') && (payload.keyObjectStorage matches '^^S3\\w{2}(OL_1_E(F|R)R|SL_1_RBT|SR_1_SRA).*')) || ((payload.productFamily == 'S3_L2_NRT') && (payload.keyObjectStorage matches '^^S3\\w{2}(OL_2_L(F|R)R|SL_2_(LST|FRP)|SR_2_LAN|SY_2_(AOD|SYN|V10|VG1)).*'))

app.preparation-worker.tasktable.routing.SL_2_FRP____A=TaskTable.PUG_SL_2_FRP.03.xml
app.preparation-worker.tasktable.routing.SL_2_FRP____B=TaskTable.PUG_SL_2_FRP.03.xml
app.preparation-worker.pdu.config.SL_2_FRP___.type=FRAME
app.preparation-worker.pdu.config.SL_2_FRP___.length-in-s=180
app.preparation-worker.pdu.config.SL_2_FRP___.gap-threshhold-in-s=3
app.preparation-worker.pdu.config.SL_2_FRP___.dyn-proc-params.facilityName=LN3
app.preparation-worker.pdu.config.SL_2_FRP___.dyn-proc-params.hardwareName=O

app.housekeep.tasktable.routing.SL_2_FRP____A=TaskTable.PUG_SL_2_FRP.03.xml
app.housekeep.tasktable.routing.SL_2_FRP____B=TaskTable.PUG_SL_2_FRP.03.xml
app.housekeep.pdu.config.SL_2_FRP___.type=FRAME
app.housekeep.pdu.config.SL_2_FRP___.length-in-s=180
app.housekeep.pdu.config.SL_2_FRP___.gap-threshhold-in-s=3
app.housekeep.pdu.config.SL_2_FRP___.dyn-proc-params.facilityName=LN3
app.housekeep.pdu.config.SL_2_FRP___.dyn-proc-params.hardwareName=O

pug-ntc

app.message-filter.filter.function.expression=((payload.productFamily == 'S3_AUX') && (payload.keyObjectStorage matches '^S3\\w{2}(AX___(BB2|FPO|FRO|OSF)_AX|(OL|SL)_(1|2)_(PCP|PLT)BAX|SY_2_(CVS|PCP|PLT|PVS)BAX).*')) || ((payload.productFamily == 'S3_L1_NTC') && (payload.keyObjectStorage matches '^^S3\\w{2}(OL_1_E(F|R)R|SL_1_RBT|SR_1_SRA).*')) || ((payload.productFamily == 'S3_L2_NTC') && (payload.keyObjectStorage matches '^^S3\\w{2}(OL_2_L(F|R)R|SL_2_(LST|FRP)|SR_2_LAN|SY_2_(AOD|SYN|V10|VG1)).*'))

app.preparation-worker.tasktable.routing.SL_2_FRP____A=TaskTable.PUG_SL_2_FRP.03.xml
app.preparation-worker.tasktable.routing.SL_2_FRP____B=TaskTable.PUG_SL_2_FRP.03.xml
app.preparation-worker.pdu.config.SL_2_FRP___.type=FRAME
app.preparation-worker.pdu.config.SL_2_FRP___.length-in-s=180
app.preparation-worker.pdu.config.SL_2_FRP___.gap-threshhold-in-s=3
app.preparation-worker.pdu.config.SL_2_FRP___.dyn-proc-params.facilityName=LN3
app.preparation-worker.pdu.config.SL_2_FRP___.dyn-proc-params.hardwareName=O

app.housekeep.tasktable.routing.SL_2_FRP____A=TaskTable.PUG_SL_2_FRP.03.xml
app.housekeep.tasktable.routing.SL_2_FRP____B=TaskTable.PUG_SL_2_FRP.03.xml
app.housekeep.pdu.config.SL_2_FRP___.type=FRAME
app.housekeep.pdu.config.SL_2_FRP___.length-in-s=180
app.housekeep.pdu.config.SL_2_FRP___.gap-threshhold-in-s=3
app.housekeep.pdu.config.SL_2_FRP___.dyn-proc-params.facilityName=LN3
app.housekeep.pdu.config.SL_2_FRP___.dyn-proc-params.hardwareName=O

The STC chain is not affected, as SL products are only part of NRT and NTC production.

[SYTHIER-ADS]

This is now applied on the configuration of the preint tests (https://github.com/COPRS/rs-config/commit/95b89d0a8cb8a81dafff9ddd668d44d16fb87020)

[Woljtek]

System_CCB_2023-w30 : To be discuss during WERUM CCB

[vgava-ads]

System_CCB_2023_w31: Delivered in the Processing Sentinel-3 v.14.0 (Refer to https://github.com/COPRS/processing-sentinel-3/releases/tag/1.14.0-rc2) To be validated by IVV/OPS team.

[LAQU156]

System_CCB_2023_w31 : To be tested with next deployment

[suberti-ads]

On configuration pug-ntc https://github.com/COPRS/processing-sentinel-3/blob/1.14.0-rc2/s3-pug-ntc/content/stream-parameters.properties gap was not set to 3 but to 0.2

app.preparation-worker.pdu.config.SL_2_FRP___.gap-threshhold-in-s=0.2

By experience we set all parameter gap-threshhold-in (for all type are set to 3) So i propose to keep this issue open with this WA.

[LAQU156]

System_CCB_2023_w33 : Validation failed, Will be delivered after phase 1

[Woljtek]

System_CCB_2023_w34: Accepted WERUM. Handled by version 1.14.0

[Woljtek]

System_CCB_2023_w34: Successfully tested with WA on OPS (fix=wa)