search

epi2me-labs / wf-human-variation

Other
105 stars 45 forks source link

Spectre CNV results not the same as QDNAseq #182

Closed charlenelawdes closed 3 months ago

charlenelawdes commented 6 months ago

Operating System

CentOS 7 DEMO.wf-human-cnv-report_qdnaseq.pdf DEMO.wf-human-cnv-report_spectre.pdf

Other Linux

No response

Workflow Version

v2.2.0

Workflow Execution

Command line (Cluster)

Other workflow execution

No response

EPI2ME Version

No response

CLI command run

EXARGS="--snp --cnv --str --phased --sv --threads 40 --ubam_map_threads 30 ubam_sort_threads 20"
BCMOD=dna_r10.4.1_e8.2_400bps_hac_prom
BAM=/home/clawrukd/wf-human-variation-demo/demo.bam
REF=/home/clawrukd/wf-human-variation-demo/demo.fasta
NAME=DEMO
OUT=/home/clawrukd/wf-human-variation-test/Spectre
APPTAINER_CONFIG=/home/clawrukd/apptainer_nextflow.config

nextflow run epi2me-labs/wf-human-variation -resume \
    $EXARGS \
    -w $WORKDIR \
    --basecaller_cfg $BCMOD \
    --bam $BAM \
    --ref $REF \
    --sample_name $NAME \
    --out_dir $OUT \
    -c $APPTAINER_CONFIG \
    -profile apptainer

nextflow run epi2me-labs/wf-human-variation -resume \
    $EXARGS \
    -w $WORKDIR \
    --use_qdnaseq \
    --basecaller_cfg $BCMOD \
    --bam $BAM \
    --ref $REF \
    --sample_name $NAME \
    --out_dir ${BASEDIR}/QDNAseq \
    -c $APPTAINER_CONFIG \
    -profile apptainer

Workflow Execution - CLI Execution Profile

None

What happened?

I ran the workflow on the demo dataset provided by this workflow, and I get different results when running --cnv with and without --use_qdnaseq. The reports also look different, with useful graphs present in the version using QDNAseq no longer present with Spectre.

When running with QDNAseq, I get a CNV for chromosome 20, but I get no CNV when running with Spectre on the same demo dataset.

I included the run output for the run with Spectre and the two reports in PDF.

Relevant log output

curl: (28) Failed to connect to www.nextflow.io port 443 after 300100 ms: Timeout was reached
N E X T F L O W  ~  version 23.10.0
Launching `https://github.com/epi2me-labs/wf-human-variation` [lonely_saha] DSL2 - revision: 5651930a05 [master]

||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-human-variation v2.2.0-g5651930
--------------------------------------------------------------------------------
Core Nextflow options
  revision        : master
  runName         : lonely_saha
  containerEngine : apptainer
  container       : [withLabel:wf_human_snp:ontresearch/wf-human-variation-snp:sha6d5c80963ce346695d98205563c87dc71fc182ff, withLabel:wf_human_sv:ontresearch/wf-human-variation-sv:sha8f02fbfaea233fe4763f7c7142696efcadab2f4d, withLabel:wf_human_mod:ontresearch/modkit:sha0253e9e9ba92aacc852ba376edefe8ff0932f71a, withLabel:wf_cnv:ontresearch/wf-cnv:sha428cb19e51370020ccf29ec2af4eead44c6a17c2, withLabel:wf_human_str:ontresearch/wf-human-variation-str:shaa2f49ce57886426516eadd4048b6fdf9c22c7437, withLabel:snpeff_annotation:ontresearch/snpeff:sha313729d816872d70b410752001a0e2654eb88807, withLabel:wf_common:ontresearch/wf-common:sha91cd87900c86f05bf36d8c77b841b8fda5ecf3aa, withLabel:spectre:ontresearch/spectre:sha15d6757d00b503344d0b1054473ffeb5e3cab79d, default:ontresearch/wf-human-variation:sha0f033fe23a576cf4da9df8d2d6745d63329a8611]
  launchDir       : /home/clawrukd/wf-human-variation-test
  workDir         : /home/clawrukd/wf-human-variation-test
  projectDir      : /home/clawrukd/.nextflow/assets/epi2me-labs/wf-human-variation
  userName        : clawrukd
  profile         : apptainer
  configFiles     : /home/clawrukd/.nextflow/assets/epi2me-labs/wf-human-variation/nextflow.config, /home/clawrukd/apptainer_nextflow.config

Workflow Options
  sv              : true
  snp             : true
  cnv             : true
  str             : true

Main options
  sample_name     : DEMO
  bam             : /home/clawrukd/wf-human-variation-test/wf-human-variation-demo/demo.bam
  ref             : /home/clawrukd/wf-human-variation-test/wf-human-variation-demo/demo.fasta
  basecaller_cfg  : dna_r10.4.1_e8.2_400bps_hac_prom
  phased          : true
  out_dir         : /home/clawrukd/wf-human-variation-test/Spectre

Multiprocessing Options
  threads         : 40
  ubam_map_threads: 30

!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-human-variation for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

--------------------------------------------------------------------------------
This is epi2me-labs/wf-human-variation v2.2.0-g5651930.
--------------------------------------------------------------------------------
executor >  local (90)
[80/732b1d] process > ingress:checkBamHeaders (1)    [100%] 1 of 1, cached: 1 ✔
[-        ] process > ingress:sortBam                -
[-        ] process > ingress:mergeBams              -
[-        ] process > ingress:catSortBams            -
[35/d5d577] process > ingress:validateIndex (1)      [100%] 1 of 1, cached: 1 ✔
[-        ] process > ingress:samtools_index         -
[04/9fb071] process > ingress:check_for_alignment... [100%] 1 of 1, cached: 1 ✔
[-        ] process > ingress:minimap2_alignment     -
[e0/b38fb9] process > getGenome (1)                  [100%] 1 of 1 ✔
[a2/7ee4f4] process > cram_cache (1)                 [100%] 1 of 1, cached: 1 ✔
[e6/4b2821] process > getAllChromosomesBed (1)       [100%] 1 of 1, cached: 1 ✔
[5a/7cd542] process > mosdepth_input (1)             [100%] 1 of 1, cached: 1 ✔
[76/6b440c] process > getVersions                    [100%] 1 of 1, cached: 1 ✔
[57/6f978b] process > getParams                      [100%] 1 of 1, cached: 1 ✔
[9a/78ba5c] process > readStats (1)                  [100%] 1 of 1, cached: 1 ✔
[95/4d56e6] process > infer_sex (1)                  [100%] 1 of 1, cached: 1 ✔
[0f/ef986a] process > makeAlignmentReport            [100%] 1 of 1, cached: 1 ✔
[-        ] process > failedQCReport                 -
[5f/d272f5] process > lookup_clair3_model (1)        [100%] 1 of 1, cached: 1 ✔
[f2/717b83] process > snp:make_chunks (1)            [100%] 1 of 1 ✔
[04/f3c28a] process > snp:pileup_variants (6)        [100%] 13 of 13 ✔
[09/7e6bda] process > snp:aggregate_pileup_varian... [100%] 1 of 1 ✔
[eb/d835ea] process > snp:select_het_snps (1)        [100%] 1 of 1 ✔
[7b/cba70a] process > snp:phase_contig (1)           [100%] 1 of 1 ✔
[fb/c63951] process > snp:get_qual_filter (1)        [100%] 1 of 1 ✔
[bb/dbf178] process > snp:create_candidates (1)      [100%] 1 of 1 ✔
[da/6a76d8] process > snp:evaluate_candidates (2)    [100%] 15 of 15 ✔
[ec/12613c] process > snp:aggregate_full_align_va... [100%] 1 of 1 ✔
[d4/303f2b] process > snp:merge_pileup_and_full_v... [100%] 1 of 1 ✔
[b3/755002] process > snp:post_clair_phase_contig... [100%] 1 of 1 ✔
[d5/f47238] process > snp:post_clair_contig_haplo... [100%] 1 of 1 ✔
[52/b10347] process > snp:cat_haplotagged_contigs... [100%] 1 of 1 ✔
[e1/819305] process > snp:aggregate_all_variants (1) [100%] 1 of 1 ✔
[1f/fe7f31] process > snp:haploblocks_snp (1)        [100%] 1 of 1 ✔
[82/ae48f0] process > sv:variantCall:sniffles2 (1)   [100%] 1 of 1 ✔
[e4/9874dc] process > sv:variantCall:filterCalls (1) [100%] 1 of 1 ✔
[d2/79af25] process > sv:variantCall:sortVCF (1)     [100%] 1 of 1 ✔
[90/c6bd8c] process > sv:annotate_sv_vcf (1)         [100%] 1 of 1 ✔
[bb/f8f63e] process > sv:runReport:getVersions       [100%] 1 of 1, cached: 1 ✔
[1c/b14102] process > sv:runReport:getParams         [100%] 1 of 1, cached: 1 ✔
[51/b15645] process > sv:runReport:report (1)        [100%] 1 of 1 ✔
[33/820432] process > output_sv (3)                  [100%] 4 of 4, cached: 1 ✔
[90/7d5eb0] process > refine_with_sv (1)             [100%] 1 of 1 ✔
[fa/a4e1eb] process > concat_refined_snp             [100%] 1 of 1 ✔
[a4/3c11a4] process > annotate_snp_vcf (1)           [100%] 1 of 1 ✔
[df/cb5d32] process > concat_snp_vcfs                [100%] 1 of 1 ✔
[09/148182] process > sift_clinvar_snp_vcf (1)       [100%] 1 of 1 ✔
[4f/aabbea] process > vcfStats                       [100%] 1 of 1 ✔
[dc/339a31] process > report_snp:getVersions         [100%] 1 of 1, cached: 1 ✔
[1f/4fdc76] process > report_snp:getParams           [100%] 1 of 1, cached: 1 ✔
[9c/67e08b] process > report_snp:makeReport (1)      [100%] 1 of 1 ✔
[28/23509d] process > output_snp (2)                 [100%] 7 of 7 ✔
[4d/a6f1a0] process > cnv_spectre:mosdepth (1)       [100%] 1 of 1 ✔
[00/ffc48f] process > cnv_spectre:callCNV (1)        [100%] 1 of 1 ✔
[0b/333333] process > cnv_spectre:bgzip_and_index... [100%] 1 of 1 ✔
[cc/168d28] process > cnv_spectre:annotate_vcf (1)   [100%] 1 of 1 ✔
[ee/e86d4a] process > cnv_spectre:getVersions        [100%] 1 of 1 ✔
[ac/d7db97] process > cnv_spectre:add_snp_tools_t... [100%] 1 of 1 ✔
[37/5f144a] process > cnv_spectre:getParams          [100%] 1 of 1 ✔
[3d/bb5333] process > cnv_spectre:makeReport (1)     [100%] 1 of 1 ✔
[53/bd5674] process > output_cnv (1)                 [100%] 2 of 2 ✔
[8f/97acc7] process > str:call_str (1)               [100%] 1 of 1 ✔
[e6/3d8e0f] process > str:annotate_repeat_expansi... [100%] 1 of 1 ✔
[86/a0fcbb] process > str:getVersions                [100%] 1 of 1, cached: 1 ✔
[cf/0b3aac] process > str:getParams                  [100%] 1 of 1, cached: 1 ✔
[51/e69caf] process > str:bam_region_filter (1)      [100%] 1 of 1 ✔
[83/17bde7] process > str:bam_read_filter (1)        [100%] 1 of 1 ✔
[46/d4a751] process > str:generate_str_content (1)   [100%] 1 of 1 ✔
[0f/ee0054] process > str:concat_str_vcfs            [100%] 1 of 1 ✔
[c0/2e44be] process > str:merge_tsv                  [100%] 1 of 1 ✔
[e4/b5e958] process > str:make_report (1)            [100%] 1 of 1 ✔
[56/6aae43] process > output_str (4)                 [100%] 4 of 4 ✔
[9f/35f14a] process > configure_jbrowse (1)          [100%] 1 of 1 ✔
[94/430e3a] process > combine_metrics_json (1)       [100%] 1 of 1 ✔
[b2/bef23c] process > publish_artifact (9)           [100%] 9 of 9, cached: 7 ✔
Completed at: 22-May-2024 17:56:20
Duration    : 31m 58s
CPU hours   : 2.8 (3.2% cached)
Succeeded   : 90
Cached      : 26

Application activity log entry

No response

Were you able to successfully run the latest version of the workflow with the demo data?

yes

Other demo data information

No response

mattdmem commented 6 months ago

Hello @charlenelawdes

Thanks for your interest in the workflow.

I ran the workflow on the demo dataset provided by this workflow, and I get different results when running --cnv with and without --use_qdnaseq. The reports also look different, with useful graphs present in the version using QDNAseq no longer present with Spectre.

We're working on adding plots to the spectre output, we were keen to get spectre into our users hands as quickly as possible and so visualisations are lagging behind somewhat. Apologies for that.

When running with QDNAseq, I get a CNV for chromosome 20, but I get no CNV when running with Spectre on the same demo dataset.

These are two very different algorithms and so differences are to be expected. It would be great to hear your feedback when you get the two tools running on real samples. Demo data is often not complete as we can't ship huge amounts of data.

Matt

vlshesketh commented 3 months ago

Hi @charlenelawdes, I'll close this for now but please feel free to re-open if you have any further questions.