GC bias data missing from CEN runs

[Yu-jinKim]

DISCLAIMER

This could affect other metrics in other assays

Description

Adriana sent me a request to plot some GC coverage data for a recent CEN run. While attempting to plot, i got errors and looking more in depth in the multiqc_data.json file for a random CEN run, i found out that GC bias data is not stored in the report_saved_raw_data key which i used to define which tools are present for each assay.

Potential fixes

Instead GC bias is stored in the report_plot_data key. A few options can be considered to fix this:

• Cry
• Adapt the code to look into the report_plot_data and get the data
  • This would require a lot of work
  • Potential duplication of data since some keys in report_plot_data contain data already parsed in report_saved_raw_data
• Not trust MultiQC and get the metrics from the individual tools
  • Even more work
  • Refactoring
  • Additional configs for TrendyQC
• Find a way to move or duplicate data from report_plot_data to report_saved_raw_data
  • Least amount of work
  • Not necessarily feasible

[Yu-jinKim]

Basically MultiQC doesn't save the data per sample because the code goes in https://github.com/MultiQC/MultiQC/blob/f0e29ea3922506383c431888282a58c1af7f98af/multiqc/modules/picard/GcBiasMetrics.py#L55 due to the header in the per sample file for GC bias which doesn't allow the summary_data_by_sample to get the data needed. As a consequence the writing of data is not triggered.

[Yu-jinKim]

MultiQC v1.14 did not have complete support for Sentieon:

• https://github.com/MultiQC/MultiQC/commit/56ea024a483dea7a61e32b93c43897b43e426bae
• https://github.com/MultiQC/MultiQC/pull/2110

This caused the GCBias Summary file to not be captured by MultiQC.

MultiQC v1.18 remedied this: https://github.com/MultiQC/MultiQC/releases/tag/v1.18

[Yu-jinKim]

I have attempted to use the latest Docker image available to see if it solved the issue. This resulted in the following modifications to be made:

• Docker for MultiQC
  • I kept getting permissions error when trying to write the report file. I ended up having to remove this: https://github.com/MultiQC/MultiQC/blob/main/Dockerfile#L38
• MultiQC config i used the v2.2.0 version of CEN config as a template
  • I had to remove this:https://github.com/eastgenomics/eggd_MultiQC_configs/blob/main/CEN/CEN_multiqc_config_v2.2.0.yaml#L36 as the new version of MultiQC uses that line as the filename (overriding the CLI arg)
• MultiQC egg app
  • The line to run docker in the eggd app (https://github.com/eastgenomics/eggd_multiqc/blob/master/src/multiqc.sh#L122) had to be changed to docker run -v /home/dnanexus:/egg $MultiQC_image multiqc /egg/"$folder_name" -c /egg/config.yaml -n /egg/${outdir}/$report_name -o /egg/${outdir} otherwise the JSON and other files were not outputted in the correct location for upload in DNAnexus and multiqc is now required by the Docker image before giving the parameters

After all that, I still have issues as the samples are not correctly recognized by the Sentieon tools: https://platform.dnanexus.com/panx/projects/GkyfKF8481PpyVvXQ3FGkFq4/data/?scope=project&id.values=file-Gp220Gj4501Ypzx60f6J7Ggq I do capture some Sentieon files but not all. So i suspect some MultiQC config shenanigans.

[Yu-jinKim]

Decision taken on 03/07/24: For v1 of TrendyQC data, go ahead with the missing GC bias data. This will be re-evaluated for a subsequent version.