umccr / data-portal-apis

UMCCR Data Portal API Backend in cloud native serverless
1 stars 0 forks source link

Fix `wgs_tumor_normal` alignment engineParameter override for many FASTQ input files #671

Closed victorskl closed 2 weeks ago

victorskl commented 1 month ago

Context:

Also known as "library 4 lanes split"

Case 1:

Case 2:

Team investigation:

See Slack thread by team on investigation:

Issue:

Upstream ICA WES engineParameter for TES; by default, the relevant flag set to auto as follows.

"tesUseInputManifest": "auto"

By changing engineParameter override to never should counter the issue.

"tesUseInputManifest": "never"

Action:

victorskl commented 1 month ago

How to add EngineParameter override to SecondaryAnalysisHelper in workflow.

Follow ctTSO scatter flag override

https://github.com/umccr/data-portal-apis/blob/7526347/data_processors/pipeline/domain/workflow.py#L243-L245

victorskl commented 1 month ago

Potential fix snippet:

        if self.type == WorkflowType.TUMOR_NORMAL:
            # <link to upstream issue pointer, if any>
            engine_params.update(tesUseInputManifest="never")

Need to run full test suite.

victorskl commented 1 month ago

In COLUMN_PI cohort, we leveraged this tesUseInputManifest = "never" flag.

https://github.com/search?q=repo%3Aumccr%2Fbiodaily%20tesUseInputManifest&type=code


Note though: This has done "Out-of-Band" [1] workflow run by directly launching at ICA WES. It involves couple of manual steps.

REF:

  1. https://github.com/umccr/data-portal-apis/blob/dev/docs/pipeline/portal_run_id.md
victorskl commented 2 weeks ago

we can enable this for all workflow types.

ditto #693 convo