search

epi2me-labs / wf-human-variation

Other
101 stars 43 forks source link

[Bug]: Singularity profile not working for demo data #43

Closed ymcki closed 1 year ago

ymcki commented 1 year ago

What happened?

I am running this nextflow script locally on my ubuntu 18.04 machine. The standard profile works as expected with the demo data. However, the process get killed halfway when I switch to singularity profile

==========

[84/5048cf] process > cnv:callCNV (1) [ 0%] 0 of 1 [ab/713084] process > cnv:getVersions [100%] 1 of 1 ✔ [e9/812d68] process > cnv:getParams [100%] 1 of 1 ✔ [- ] process > cnv:makeReport - [- ] process > output_cnv - [- ] process > str:call_str - [- ] process > str:annotate_repeat_expans... - [b4/d4c2ff] process > str:getVersions [100%] 1 of 1 ✔ [70/58ac55] process > str:getParams [100%] 1 of 1 ✔ [- ] process > str:merge_vcf - [- ] process > str:merge_tsv - [- ] process > str:make_report - [- ] process > output_str - [e6/b0870d] process > configure_jbrowse (1) [100%] 1 of 1 ✔ [bf/7b1f6f] process > publish_artifact (7) [100%] 9 of 9 Pulling Singularity image docker://ontresearch/wf-human-variation-methyl:sha44a13bcf48db332b2277bb9f95b56d64e393a1d5 [cache /home/abc/ont/images/ontresearch-wf-human-variation-methyl-sha44a13bcf48db332b2277bb9f95b56d64e393a1d5.img] Killed

My command line looks like: ./nextflow run wf-human-variation -w output/workspace -profile singularity --snp --sv --methyl --cnv --mapula --str --bam demo_data/demo.bam --bed demo_data/demo.bed --ref demo_data/demo.fasta --basecaller_cfg 'dna_r10.4.1_e8.2_400bps_hac@v3.5.2' --remora_cfg 'dna_r10.4.1_e8.2_400bps_hac@v3.5.2_5mCG@v2' --sample_name MY_SAMPLE --outdir output

What is the problem here? Can it be due to some permission setup at Singularity?

Operating System

ubuntu 18.04

Workflow Execution

Command line

Workflow Execution - EPI2ME Labs Versions

No response

Workflow Execution - CLI Execution Profile

None

Workflow Version

1.5.2

Relevant log output

Full Log:

Jun-05 11:06:58.750 [main] DEBUG nextflow.cli.Launcher - $> nextflow run wf-human-variation -w output/workspace -profile singularity --snp --sv --methyl --cnv --mapula --str --bam demo_data/demo.bam --bed demo_data/demo.bed --ref demo_data/demo.fasta --basecaller_cfg dna_r10.4.1_e8.2_400bps_hac@v3.5.2 --remora_cfg dna_r10.4.1_e8.2_400bps_hac@v3.5.2_5mCG@v2 --sample_name MY_SAMPLE --outdir output
Jun-05 11:06:58.857 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 23.04.1
Jun-05 11:06:58.875 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/home/abc/.nextflow/plugins; core-plugins: nf-amazon@1.16.2,nf-azure@1.0.1,nf-codecommit@0.1.4,nf-console@1.0.5,nf-ga4gh@1.0.5,nf-google@1.7.3,nf-tower@1.5.12,nf-wave@0.8.2
Jun-05 11:06:58.887 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Jun-05 11:06:58.888 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Jun-05 11:06:58.891 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Jun-05 11:06:58.901 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Jun-05 11:07:01.261 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /home/abc/ont/wf-human-variation/nextflow.config
Jun-05 11:07:01.262 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/abc/ont/wf-human-variation/nextflow.config
Jun-05 11:07:01.272 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `singularity`
Jun-05 11:07:01.672 [main] DEBUG nextflow.config.ConfigBuilder - Available config profiles: [standard, conda, singularity, awsbatch, local]
Jun-05 11:07:01.742 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declararion
Jun-05 11:07:01.762 [main] INFO  nextflow.cli.CmdRun - Launching `wf-human-variation/main.nf` [scruffy_hoover] DSL2 - revision: 9dd4d8b0f4
Jun-05 11:07:01.763 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Jun-05 11:07:01.763 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[]
Jun-05 11:07:01.770 [main] DEBUG nextflow.secret.LocalSecretsProvider - Secrets store: /home/abc/.nextflow/secrets/store.json
Jun-05 11:07:01.773 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@470d183] - activable => nextflow.secret.LocalSecretsProvider@470d183
Jun-05 11:07:01.838 [main] DEBUG nextflow.Session - Session UUID: efbfc584-1790-49b0-b83a-b1e07b81a099
Jun-05 11:07:01.838 [main] DEBUG nextflow.Session - Run name: scruffy_hoover
Jun-05 11:07:01.839 [main] DEBUG nextflow.Session - Executor pool size: 256
Jun-05 11:07:01.849 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=768; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Jun-05 11:07:01.880 [main] DEBUG nextflow.cli.CmdRun -
  Version: 23.04.1 build 5866
  Created: 15-04-2023 06:51 UTC (14:51 HKST)
  System: Linux 5.19.0-38-generic
  Runtime: Groovy 3.0.16 on OpenJDK 64-Bit Server VM 11.0.19+7-post-Ubuntu-0ubuntu122.04.1
  Encoding: UTF-8 (UTF-8)
  Process: 330081@cpu1-anf-hk4 [172.16.2.100]
  CPUs: 256 - Mem: 2 TB (556 GB) - Swap: 0 (0)
Jun-05 11:07:01.969 [main] DEBUG nextflow.Session - Work-dir: /home/abc/ont/output/workspace [nfs]
Jun-05 11:07:01.998 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Jun-05 11:07:02.037 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Jun-05 11:07:02.234 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Jun-05 11:07:02.255 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 257; maxThreads: 1000
Jun-05 11:07:02.405 [main] DEBUG nextflow.Session - Session start
Jun-05 11:07:02.411 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow started -- trace file: /home/abc/ont/output/execution/trace.txt
Jun-05 11:07:02.423 [main] DEBUG nextflow.Session - Using default localLib path: /home/abc/ont/wf-human-variation/lib
Jun-05 11:07:02.426 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/abc/ont/wf-human-variation/lib
Jun-05 11:07:02.426 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/abc/ont/wf-human-variation/lib/nfcore_external_java_deps.jar
Jun-05 11:07:07.850 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Jun-05 11:07:20.552 [main] WARN  nextflow.Nextflow - Found unexpected parameters:
* --outdir: output
Jun-05 11:07:20.553 [main] INFO  nextflow.Nextflow - - ^[[2mIgnore this warning: params.schema_ignore_params = "outdir" ^[[0m
Jun-05 11:07:20.567 [main] INFO  nextflow.Nextflow -
^[[0;92m||||||||||   ^[[0m^[[2m_____ ____ ___ ____  __  __ _____      _       _
^[[0;92m||||||||||  ^[[0m^[[2m| ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
^[[0;33m|||||       ^[[0m^[[2m|  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
^[[0;33m|||||       ^[[0m^[[2m| |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
^[[0;94m||||||||||  ^[[0m^[[2m|_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
^[[0;94m||||||||||  ^[[0m^[[1mwf-human-variation v1.5.2^[[0m
^[[2m--------------------------------------------------------------------------------^[[0m
^[[1mCore Nextflow options^[[0m
  ^[[0;34mrunName            : ^[[0;32mscruffy_hoover^[[0m
  ^[[0;34mcontainerEngine    : ^[[0;32msingularity^[[0m
  ^[[0;34mcontainer          : ^[[0;32montresearch/wf-human-variation:sha2f95a5aa3a56047fbd3f3bcf427d114ca26f182d^[[0m
  ^[[0;34mlaunchDir          : ^[[0;32m/home/abc/ont^[[0m
  ^[[0;34mworkDir            : ^[[0;32m/home/abc/ont/output/workspace^[[0m
  ^[[0;34mprojectDir         : ^[[0;32m/home/abc/ont/wf-human-variation^[[0m
  ^[[0;34muserName           : ^[[0;32meymchan^[[0m
  ^[[0;34mprofile            : ^[[0;32msingularity^[[0m
  ^[[0;34mconfigFiles        : ^[[0;32m/home/abc/ont/wf-human-variation/nextflow.config^[[0m

^[[1mWorkflow Options^[[0m
  ^[[0;34msv                 : ^[[0;32mtrue^[[0m
  ^[[0;34msnp                : ^[[0;32mtrue^[[0m
  ^[[0;34mcnv                : ^[[0;32mtrue^[[0m
  ^[[0;34mmethyl             : ^[[0;32mtrue^[[0m
  ^[[0;34mstr                : ^[[0;32mtrue^[[0m
  ^[[0;34mmapula             : ^[[0;32mtrue^[[0m

^[[1mInput Options^[[0m
  ^[[0;34mbam                : ^[[0;32mdemo_data/demo.bam^[[0m
  ^[[0;34mref                : ^[[0;32mdemo_data/demo.fasta^[[0m
  ^[[0;34mbasecaller_cfg     : ^[[0;32mdna_r10.4.1_e8.2_400bps_hac@v3.5.2^[[0m
  ^[[0;34mbed                : ^[[0;32mdemo_data/demo.bed^[[0m

^[[1mBasecalling options^[[0m
  ^[[0;34mremora_cfg         : ^[[0;32mdna_r10.4.1_e8.2_400bps_hac@v3.5.2_5mCG@v2^[[0m

^[[1mOutput Options^[[0m
  ^[[0;34msample_name        : ^[[0;32mMY_SAMPLE^[[0m

^[[1mMultiprocessing Options^[[0m
  ^[[0;34mubam_map_threads   : ^[[0;32m8^[[0m
  ^[[0;34mubam_sort_threads  : ^[[0;32m3^[[0m
  ^[[0;34mubam_bam2fq_threads: ^[[0;32m1^[[0m

^[[1mOther parameters^[[0m
  ^[[0;34mprocess_label      : ^[[0;32mwfdefault^[[0m

!! Only displaying parameters that differ from the pipeline defaults !!
^[[2m--------------------------------------------------------------------------------^[[0m
If you use epi2me-labs/wf-human-variation for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

^[[2m--------------------------------------------------------------------------------^[[0m
This is epi2me-labs/wf-human-variation v1.5.2.
^[[2m--------------------------------------------------------------------------------^[[0m
Jun-05 11:07:22.077 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.077 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.083 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Jun-05 11:07:22.137 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=96; memory=256 GB; capacity=256; pollInterval=100ms; dumpInterval=5m
Jun-05 11:07:22.284 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.285 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.355 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.356 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.364 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.364 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.433 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.433 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.448 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name readStats
Jun-05 11:07:22.449 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.449 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.457 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.457 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.460 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.460 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.462 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.463 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.465 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.497 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.497 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.571 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.572 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.579 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_sv` matches labels `wf_human_sv` for process with name sv:variantCall:filterBam
Jun-05 11:07:22.579 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.580 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.582 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_sv` matches labels `wf_human_sv` for process with name sv:variantCall:sniffles2
Jun-05 11:07:22.583 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.583 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.586 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.586 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.588 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_sv` matches labels `wf_human_sv` for process with name sv:variantCall:sortVCF
Jun-05 11:07:22.588 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.589 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.594 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_sv` matches labels `wf_human_sv` for process with name sv:variantCall:indexVCF
Jun-05 11:07:22.594 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.594 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.599 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_sv` matches labels `wf_human_sv` for process with name sv:runReport:getVersions
Jun-05 11:07:22.599 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.599 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.601 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_sv` matches labels `wf_human_sv` for process with name sv:runReport:getParams
Jun-05 11:07:22.601 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.601 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.606 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.606 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.613 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_sv` matches labels `wf_human_sv` for process with name output_sv
Jun-05 11:07:22.613 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.613 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.615 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name lookup_clair3_model
Jun-05 11:07:22.615 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.615 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.635 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name snp:make_chunks
Jun-05 11:07:22.635 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.635 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.657 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name snp:pileup_variants
Jun-05 11:07:22.666 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.666 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.668 [main] DEBUG nextflow.processor.TaskProcessor - Creating *combiner* operator for each param(s) at index(es): [0]
Jun-05 11:07:22.676 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name snp:aggregate_pileup_variants
Jun-05 11:07:22.676 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.676 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.680 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name snp:select_het_snps
Jun-05 11:07:22.684 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.685 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.685 [main] DEBUG nextflow.processor.TaskProcessor - Creating *combiner* operator for each param(s) at index(es): [0]
Jun-05 11:07:22.695 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name snp:phase_contig_haplotag
Jun-05 11:07:22.695 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.695 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.736 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name snp:merge_haplotagged_contigs
Jun-05 11:07:22.736 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.736 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.738 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name snp:get_qual_filter
Jun-05 11:07:22.739 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.739 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.741 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name snp:create_candidates
Jun-05 11:07:22.742 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.742 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.743 [main] DEBUG nextflow.processor.TaskProcessor - Creating *combiner* operator for each param(s) at index(es): [0]
Jun-05 11:07:22.760 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name snp:evaluate_candidates
Jun-05 11:07:22.760 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.760 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.770 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name snp:aggregate_full_align_variants
Jun-05 11:07:22.770 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.770 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.777 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name snp:merge_pileup_and_full_vars
Jun-05 11:07:22.779 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.779 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.779 [main] DEBUG nextflow.processor.TaskProcessor - Creating *combiner* operator for each param(s) at index(es): [0]
Jun-05 11:07:22.835 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name snp:aggregate_all_variants
Jun-05 11:07:22.836 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.836 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.839 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name snp:refine_with_sv
Jun-05 11:07:22.840 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.840 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.842 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name snp:getVersions
Jun-05 11:07:22.842 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.843 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.844 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name snp:getParams
Jun-05 11:07:22.845 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.845 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.846 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name snp:vcfStats
Jun-05 11:07:22.846 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.846 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.849 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.849 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.861 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_snp` matches labels `wf_human_snp` for process with name output_snp
Jun-05 11:07:22.861 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.861 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.869 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_methyl` matches labels `wf_human_methyl` for process with name methyl:modbam2bed
Jun-05 11:07:22.869 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.869 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.875 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_cnv` matches labels `wf_cnv` for process with name cnv:callCNV
Jun-05 11:07:22.875 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.875 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.877 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_cnv` matches labels `wf_cnv` for process with name cnv:getVersions
Jun-05 11:07:22.877 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.877 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.878 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_cnv` matches labels `wf_cnv` for process with name cnv:getParams
Jun-05 11:07:22.878 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.878 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.881 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_cnv` matches labels `wf_cnv` for process with name cnv:makeReport
Jun-05 11:07:22.881 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.881 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.883 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_cnv` matches labels `wf_cnv` for process with name output_cnv
Jun-05 11:07:22.883 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.883 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.887 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_str` matches labels `wf_human_str` for process with name str:call_str
Jun-05 11:07:22.887 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.887 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.932 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_str` matches labels `wf_human_str` for process with name str:annotate_repeat_expansions
Jun-05 11:07:22.932 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.932 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.934 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_str` matches labels `wf_human_str` for process with name str:getVersions
Jun-05 11:07:22.935 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.935 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.936 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_str` matches labels `wf_human_str` for process with name str:getParams
Jun-05 11:07:22.936 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.936 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.943 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.943 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.949 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.949 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.954 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_str` matches labels `wf_human_str` for process with name str:make_report
Jun-05 11:07:22.955 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.955 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.959 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_human_str` matches labels `wf_human_str` for process with name output_str
Jun-05 11:07:22.959 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.959 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.961 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.961 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:22.983 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jun-05 11:07:22.983 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jun-05 11:07:23.033 [main] DEBUG nextflow.Session - Workflow process names [dsl2]: filterBam, get_qual_filter, intersectBedWithTruthset, makeAlignmentReport, snp:evaluate_candidates, create_candidates, index_ref_gzi, snp:getParams, makeReport, str:annotate_repeat_expansions, sv:runReport:getVersions, str:merge_vcf, sv:runReport:report, output_snp, bam_ingress:minimap2_ubam, merge_vcf, bam_ingress:check_for_alignment, post_clair_phase_contig, getAllChromosomesBed, output_cnv, merge_pileup_and_full_vars, snp:pileup_variants, refine_with_sv, sv:variantCall:indexVCF, mapula, readStats, evaluate_candidates, snp:phase_contig_haplotag, aggregate_pileup_variants, lookup_clair3_model, index_ref_fai, filterCalls, merge_fail_calls, cnv:callCNV, failedQCReport, snp:get_qual_filter, call_str, vcfStats, getGenome, snp:select_het_snps, snp:getVersions, snp:vcfStats, aggregate_all_variants, decompress_ref, report, snp:aggregate_all_variants, str:getVersions, modbam2bed, sv:variantCall:filterBam, snp:merge_pileup_and_full_vars, sortVCF, sv:variantCall:sniffles2, str:merge_tsv, indexVCF, filterBenchmarkVcf, hap, select_het_snps, make_mmi, getVersions, callCNV, snp:merge_haplotagged_contigs, pileup_variants, str:call_str, methyl:modbam2bed, snp:makeReport, phase_contig, cnv:makeReport, aggregate_full_align_variants, merge_tsv, snp:refine_with_sv, minimap2_ubam, merge_haplotagged_contigs, phase_contig_haplotag, configure_jbrowse, check_for_alignment, samtools_index, getParams, snp:aggregate_full_align_variants, make_report, annotate_repeat_expansions, sv:runReport:getParams, sniffles2, output_sv, cnv:getVersions, dorado_align, str:make_report, get_coverage, truvari, publish_artifact, sv:variantCall:sortVCF, make_chunks, snp:create_candidates, output_str, cram_cache, sv:variantCall:filterCalls, str:getParams, merge_calls, merge_pass_calls, dorado, mosdepth, snp:make_chunks, snp:aggregate_pileup_variants, cnv:getParams
Jun-05 11:07:23.035 [main] DEBUG nextflow.Session - Igniting dataflow network (71)
Jun-05 11:07:23.047 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > bam_ingress:check_for_alignment
Jun-05 11:07:23.051 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > bam_ingress:minimap2_ubam
Jun-05 11:07:23.056 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > getGenome
Jun-05 11:07:23.065 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > cram_cache
Jun-05 11:07:23.075 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > mosdepth
Jun-05 11:07:23.077 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > readStats
Jun-05 11:07:23.082 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > mapula
Jun-05 11:07:23.082 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > getVersions
Jun-05 11:07:23.085 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > getParams
Jun-05 11:07:23.135 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > get_coverage
Jun-05 11:07:23.139 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > failedQCReport
Jun-05 11:07:23.145 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > makeAlignmentReport
Jun-05 11:07:23.147 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > sv:variantCall:filterBam
Jun-05 11:07:23.147 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > sv:variantCall:sniffles2
Jun-05 11:07:23.148 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > sv:variantCall:filterCalls
Jun-05 11:07:23.149 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > sv:variantCall:sortVCF
Jun-05 11:07:23.149 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > sv:variantCall:indexVCF
Jun-05 11:07:23.150 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > sv:runReport:getVersions
Jun-05 11:07:23.152 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > sv:runReport:getParams
Jun-05 11:07:23.152 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > sv:runReport:report
Jun-05 11:07:23.157 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > output_sv
Jun-05 11:07:23.162 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > lookup_clair3_model
Jun-05 11:07:23.164 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:make_chunks
Jun-05 11:07:23.168 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:pileup_variants
Jun-05 11:07:23.170 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:pileup_variants
Jun-05 11:07:23.174 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:aggregate_pileup_variants
Jun-05 11:07:23.177 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:select_het_snps
Jun-05 11:07:23.179 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:select_het_snps
Jun-05 11:07:23.180 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:phase_contig_haplotag
Jun-05 11:07:23.235 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:merge_haplotagged_contigs
Jun-05 11:07:23.238 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:get_qual_filter
Jun-05 11:07:23.240 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:create_candidates
Jun-05 11:07:23.240 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:create_candidates
Jun-05 11:07:23.256 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:evaluate_candidates
Jun-05 11:07:23.257 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:aggregate_full_align_variants
Jun-05 11:07:23.257 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:merge_pileup_and_full_vars
Jun-05 11:07:23.260 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:merge_pileup_and_full_vars
Jun-05 11:07:23.262 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:aggregate_all_variants
Jun-05 11:07:23.264 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:refine_with_sv
Jun-05 11:07:23.266 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:getVersions
Jun-05 11:07:23.269 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:getParams
Jun-05 11:07:23.278 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:vcfStats
Jun-05 11:07:23.333 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > snp:makeReport
Jun-05 11:07:23.339 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > output_snp
Jun-05 11:07:23.361 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > methyl:modbam2bed
Jun-05 11:07:23.363 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > cnv:callCNV
Jun-05 11:07:23.364 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > cnv:getVersions
Jun-05 11:07:23.364 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > cnv:getParams
Jun-05 11:07:23.365 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > cnv:makeReport
Jun-05 11:07:23.368 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > output_cnv
Jun-05 11:07:23.371 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > str:call_str
Jun-05 11:07:23.373 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > str:annotate_repeat_expansions
Jun-05 11:07:23.379 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > str:getVersions
Jun-05 11:07:23.381 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > str:getParams
Jun-05 11:07:23.382 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > str:merge_vcf
Jun-05 11:07:23.433 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > str:merge_tsv
Jun-05 11:07:23.450 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > str:make_report
Jun-05 11:07:23.453 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > output_str
Jun-05 11:07:23.453 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > configure_jbrowse
Jun-05 11:07:23.455 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > publish_artifact
Jun-05 11:07:23.459 [main] DEBUG nextflow.script.ScriptRunner - > Awaiting termination
Jun-05 11:07:23.462 [main] DEBUG nextflow.Session - Session await
Jun-05 11:07:23.739 [Actor Thread 253] INFO  nextflow.container.SingularityCache - Pulling Singularity image docker://ontresearch/wf-human-variation-str:sha8a8df7d40e0c9125ed43b347811afd14a077f94e [cache /home/abc/ont/images/ontresearch-wf-human-variation-str-sha8a8df7d40e0c9125ed43b347811afd14a077f94e.img]
Jun-05 11:07:23.742 [Actor Thread 255] INFO  nextflow.container.SingularityCache - Pulling Singularity image docker://ontresearch/wf-human-variation-snp:sha7484ad4bf3f6fe7fdcfdb8e323bf429d4f8e5138 [cache /home/abc/ont/images/ontresearch-wf-human-variation-snp-sha7484ad4bf3f6fe7fdcfdb8e323bf429d4f8e5138.img]
Jun-05 11:07:23.739 [Actor Thread 257] INFO  nextflow.container.SingularityCache - Pulling Singularity image docker://ontresearch/wf-cnv:sha70f31f182dcfb71c0769d835e393b98a60802474 [cache /home/abc/ont/images/ontresearch-wf-cnv-sha70f31f182dcfb71c0769d835e393b98a60802474.img]
Jun-05 11:07:23.741 [Actor Thread 256] INFO  nextflow.container.SingularityCache - Pulling Singularity image docker://ontresearch/wf-human-variation:sha2f95a5aa3a56047fbd3f3bcf427d114ca26f182d [cache /home/abc/ont/images/ontresearch-wf-human-variation-sha2f95a5aa3a56047fbd3f3bcf427d114ca26f182d.img]
Jun-05 11:07:23.739 [Actor Thread 252] INFO  nextflow.container.SingularityCache - Pulling Singularity image docker://ontresearch/wf-human-variation-sv:sha8e7e606633fb2b06cc9b84d0e21c22f07f8f8268 [cache /home/abc/ont/images/ontresearch-wf-human-variation-sv-sha8e7e606633fb2b06cc9b84d0e21c22f07f8f8268.img]
Jun-05 11:07:23.739 [Actor Thread 252] INFO  nextflow.container.SingularityCache - Pulling Singularity image docker://ontresearch/wf-human-variation-sv:sha8e7e606633fb2b06cc9b84d0e21c22f07f8f8268 [cache /home/abc/ont/images/ontresearch-wf-human-variation-sv-sha8e7e606633fb2b06cc9b84d0e21c22f07f8f8268.img]
Jun-05 11:07:34.023 [Actor Thread 252] DEBUG nextflow.container.SingularityCache - Singularity pull complete image=docker://ontresearch/wf-human-variation-sv:sha8e7e606633fb2b06cc9b84d0e21c22f07f8f8268 path=/home/abc/ont/images/ontresearch-wf-human-variation-sv-sha8e7e606633fb2b06cc9b84d0e21c22f07f8f8268.img
Jun-05 11:07:34.208 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:34.233 [Task submitter] INFO  nextflow.Session - [44/2f0d7d] Submitted process > sv:runReport:getVersions
Jun-05 11:07:34.243 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:34.244 [Task submitter] INFO  nextflow.Session - [85/fa0253] Submitted process > sv:runReport:getParams
Jun-05 11:07:34.859 [Actor Thread 256] DEBUG nextflow.container.SingularityCache - Singularity pull complete image=docker://ontresearch/wf-human-variation:sha2f95a5aa3a56047fbd3f3bcf427d114ca26f182d path=/home/abc/ont/images/ontresearch-wf-human-variation-sha2f95a5aa3a56047fbd3f3bcf427d114ca26f182d.img
Jun-05 11:07:34.892 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:34.892 [Task submitter] INFO  nextflow.Session - [f2/0740b6] Submitted process > getParams
Jun-05 11:07:34.936 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:34.937 [Task submitter] INFO  nextflow.Session - [e1/a9b01e] Submitted process > getVersions
Jun-05 11:07:34.947 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:34.948 [Task submitter] INFO  nextflow.Session - [2c/48e303] Submitted process > publish_artifact (1)
Jun-05 11:07:34.958 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:34.959 [Task submitter] INFO  nextflow.Session - [35/32f737] Submitted process > bam_ingress:check_for_alignment (1)
Jun-05 11:07:35.039 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:35.039 [Task submitter] INFO  nextflow.Session - [a0/a85b73] Submitted process > cram_cache (1)
Jun-05 11:07:35.902 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 6; name: sv:runReport:getParams; status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/85/fa02535ea7d465a73bc12c8018eb75]
Jun-05 11:07:37.785 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 4; name: getParams; status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/f2/0740b60d56c74522288a0be3ea9121]
Jun-05 11:07:37.792 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 14; name: publish_artifact (1); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/2c/48e30324c0eb7f371deff93eea571b]
Jun-05 11:07:37.833 [Task monitor] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'PublishDir' minSize=10; maxSize=768; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Jun-05 11:07:37.935 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 2; name: getVersions; status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/e1/a9b01ef1521f09748f61eba26d80e4]
Jun-05 11:07:42.568 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 3; name: cram_cache (1); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/a0/a85b7323e56bfe58b2230db7c0c20d]
Jun-05 11:07:42.602 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:42.603 [Task submitter] INFO  nextflow.Session - [1e/ce2977] Submitted process > publish_artifact (2)
Jun-05 11:07:42.627 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:42.627 [Task submitter] INFO  nextflow.Session - [93/4048b4] Submitted process > publish_artifact (4)
Jun-05 11:07:42.649 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:42.650 [Task submitter] INFO  nextflow.Session - [ad/85ede9] Submitted process > publish_artifact (3)
Jun-05 11:07:44.855 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 5; name: sv:runReport:getVersions; status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/44/2f0d7dfed15e955a65ac5a3f409caf]
Jun-05 11:07:44.955 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 15; name: publish_artifact (2); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/1e/ce297779c6cea8a25f74f811fa1a3e]
Jun-05 11:07:44.986 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 17; name: publish_artifact (4); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/93/4048b4370e1ed1afd984696faa3533]
Jun-05 11:07:44.992 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 16; name: publish_artifact (3); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/ad/85ede987866d24b3c445d2af728abd]
Jun-05 11:07:47.486 [Actor Thread 257] DEBUG nextflow.container.SingularityCache - Singularity pull complete image=docker://ontresearch/wf-cnv:sha70f31f182dcfb71c0769d835e393b98a60802474 path=/home/abc/ont/images/ontresearch-wf-cnv-sha70f31f182dcfb71c0769d835e393b98a60802474.img
Jun-05 11:07:47.588 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:47.589 [Task submitter] INFO  nextflow.Session - [e9/812d68] Submitted process > cnv:getParams
Jun-05 11:07:47.598 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:47.599 [Task submitter] INFO  nextflow.Session - [ab/713084] Submitted process > cnv:getVersions
Jun-05 11:07:47.600 [Actor Thread 255] DEBUG nextflow.container.SingularityCache - Singularity pull complete image=docker://ontresearch/wf-human-variation-snp:sha7484ad4bf3f6fe7fdcfdb8e323bf429d4f8e5138 path=/home/abc/ont/images/ontresearch-wf-human-variation-snp-sha7484ad4bf3f6fe7fdcfdb8e323bf429d4f8e5138.img
Jun-05 11:07:47.617 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:47.617 [Task submitter] INFO  nextflow.Session - [f0/9e3529] Submitted process > snp:getParams
Jun-05 11:07:47.626 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:47.627 [Task submitter] INFO  nextflow.Session - [75/721f17] Submitted process > lookup_clair3_model (1)
Jun-05 11:07:47.639 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:47.639 [Task submitter] INFO  nextflow.Session - [42/76e90c] Submitted process > snp:getVersions
Jun-05 11:07:49.172 [Actor Thread 253] DEBUG nextflow.container.SingularityCache - Singularity pull complete image=docker://ontresearch/wf-human-variation-str:sha8a8df7d40e0c9125ed43b347811afd14a077f94e path=/home/abc/ont/images/ontresearch-wf-human-variation-str-sha8a8df7d40e0c9125ed43b347811afd14a077f94e.img
Jun-05 11:07:49.229 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 11; name: cnv:getParams; status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/e9/812d6815a34cb4e863c78750398bfd]
Jun-05 11:07:49.241 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 9; name: snp:getParams; status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/f0/9e352954c77ad6f661985a1cd33482]
Jun-05 11:07:49.245 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 8; name: snp:getVersions; status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/42/76e90c84f89f3f6ce4f61ce9471b2c]
Jun-05 11:07:49.248 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:49.248 [Task submitter] INFO  nextflow.Session - [b4/d4c2ff] Submitted process > str:getVersions
Jun-05 11:07:49.257 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:49.258 [Task submitter] INFO  nextflow.Session - [70/58ac55] Submitted process > str:getParams
Jun-05 11:07:51.184 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 13; name: str:getParams; status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/70/58ac5580f48c93632d2fd6bbc0717e]
Jun-05 11:07:51.428 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 7; name: lookup_clair3_model (1); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/75/721f17948b86736cccacf9cfc130ac]
Jun-05 11:07:51.553 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 10; name: cnv:getVersions; status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/ab/71308425ded8836f7f4a61333a59a8]
Jun-05 11:07:56.127 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 12; name: str:getVersions; status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/b4/d4c2ff262b71c6605318577a560885]
Jun-05 11:07:58.484 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: bam_ingress:check_for_alignment (1); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/35/32f7375606f90df1aecd0edb5c3940]
Jun-05 11:07:58.566 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:58.566 [Task submitter] INFO  nextflow.Session - [97/c24d5f] Submitted process > mapula (1)
Jun-05 11:07:58.575 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:58.575 [Task submitter] INFO  nextflow.Session - [3f/1baaa8] Submitted process > getGenome (1)
Jun-05 11:07:58.584 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:58.585 [Task submitter] INFO  nextflow.Session - [96/27ef7e] Submitted process > readStats (1)
Jun-05 11:07:58.593 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:58.593 [Task submitter] INFO  nextflow.Session - [fd/4eb2be] Submitted process > mosdepth (1)
Jun-05 11:07:58.636 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:07:58.637 [Task submitter] INFO  nextflow.Session - [e6/b0870d] Submitted process > configure_jbrowse (1)
Jun-05 11:08:00.231 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 20; name: getGenome (1); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/3f/1baaa8ca541d67c4d9800dabca083a]
Jun-05 11:08:22.591 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 19; name: configure_jbrowse (1); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/e6/b0870dc0b401b6a7d0a4f9ab69c462]
Jun-05 11:08:22.678 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:08:22.678 [Task submitter] INFO  nextflow.Session - [79/5fc39a] Submitted process > publish_artifact (5)
Jun-05 11:08:23.464 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 23; name: publish_artifact (5); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/79/5fc39af5ddd7bea36eb60596c657ff]
Jun-05 11:09:00.926 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 21; name: readStats (1); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/96/27ef7eda889f85587b0585654d9036]
Jun-05 11:09:00.951 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:00.952 [Task submitter] INFO  nextflow.Session - [8b/1346e4] Submitted process > publish_artifact (6)
Jun-05 11:09:01.764 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 22; name: mosdepth (1); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/fd/4eb2be8c4bdf0d6d08b0a771a3f5b3]
Jun-05 11:09:01.802 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process `mosdepth (1)` is unable to find [UnixPath]: `/home/abc/ont/output/workspace/fd/4eb2be8c4bdf0d6d08b0a771a3f5b3/MY_SAMPLE.per-base.bed.gz` (pattern: `MY_SAMPLE.per-base.bed.gz`)
Jun-05 11:09:01.964 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mosdepth > Skipping output binding because one or more optional files are missing: fileoutparam<2>
Jun-05 11:09:01.969 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 24; name: publish_artifact (6); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/8b/1346e4b480bb2357563f40bbdf5595]
Jun-05 11:09:01.995 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:01.995 [Task submitter] INFO  nextflow.Session - [47/48e98e] Submitted process > publish_artifact (8)
Jun-05 11:09:02.012 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:02.013 [Task submitter] INFO  nextflow.Session - [5e/c4a9e7] Submitted process > publish_artifact (9)
Jun-05 11:09:02.045 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:02.046 [Task submitter] INFO  nextflow.Session - [bf/7b1f6f] Submitted process > publish_artifact (7)
Jun-05 11:09:02.061 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:02.061 [Task submitter] INFO  nextflow.Session - [ac/6d11c8] Submitted process > get_coverage (1)
Jun-05 11:09:02.896 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 27; name: publish_artifact (8); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/47/48e98e89fef7da30c8b96d23617158]
Jun-05 11:09:03.034 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 28; name: publish_artifact (9); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/5e/c4a9e708d30d12257e9fe664b94216]
Jun-05 11:09:03.083 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 25; name: get_coverage (1); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/ac/6d11c8c6db41bfbda0914d4d7825e2]
Jun-05 11:09:03.106 [Actor Thread 255] INFO  nextflow.container.SingularityCache - Pulling Singularity image docker://ontresearch/wf-human-variation-methyl:sha44a13bcf48db332b2277bb9f95b56d64e393a1d5 [cache /home/abc/ont/images/ontresearch-wf-human-variation-methyl-sha44a13bcf48db332b2277bb9f95b56d64e393a1d5.img]
Jun-05 11:09:03.108 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 26; name: publish_artifact (7); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/bf/7b1f6faa60fa61ec4352097ff9684e]
Jun-05 11:09:03.140 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:03.140 [Task submitter] INFO  nextflow.Session - [f2/94af5e] Submitted process > makeAlignmentReport
Jun-05 11:09:03.150 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:03.150 [Task submitter] INFO  nextflow.Session - [d6/920d46] Submitted process > sv:variantCall:filterBam (1)
Jun-05 11:09:03.161 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:03.162 [Task submitter] INFO  nextflow.Session - [84/5048cf] Submitted process > cnv:callCNV (1)
Jun-05 11:09:03.240 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:03.241 [Task submitter] INFO  nextflow.Session - [3c/671e18] Submitted process > snp:make_chunks (1)
Jun-05 11:09:07.645 [Actor Thread 255] DEBUG nextflow.container.SingularityCache - Singularity pull complete image=docker://ontresearch/wf-human-variation-methyl:sha44a13bcf48db332b2277bb9f95b56d64e393a1d5 path=/home/abc/ont/images/ontresearch-wf-human-variation-methyl-sha44a13bcf48db332b2277bb9f95b56d64e393a1d5.img
Jun-05 11:09:07.739 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:07.740 [Task submitter] INFO  nextflow.Session - [b2/22438f] Submitted process > methyl:modbam2bed (1)
Jun-05 11:09:10.636 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 31; name: snp:make_chunks (1); status: COMPLETED; exit: 0; error: -; workDir: /home/abc/ont/output/workspace/3c/671e184148e20be6085d1b89f52396]
Jun-05 11:09:11.642 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:11.643 [Task submitter] INFO  nextflow.Session - [3c/831fd3] Submitted process > snp:pileup_variants (9)
Jun-05 11:09:11.658 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:11.659 [Task submitter] INFO  nextflow.Session - [97/383f29] Submitted process > snp:pileup_variants (5)
Jun-05 11:09:11.741 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:11.741 [Task submitter] INFO  nextflow.Session - [1e/f8c047] Submitted process > snp:pileup_variants (10)
Jun-05 11:09:11.750 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:11.750 [Task submitter] INFO  nextflow.Session - [2a/83324e] Submitted process > snp:pileup_variants (2)
Jun-05 11:09:11.835 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:11.836 [Task submitter] INFO  nextflow.Session - [03/601767] Submitted process > snp:pileup_variants (11)
Jun-05 11:09:11.852 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:11.853 [Task submitter] INFO  nextflow.Session - [02/3e92ed] Submitted process > snp:pileup_variants (4)
Jun-05 11:09:11.941 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:12.031 [Task submitter] INFO  nextflow.Session - [0c/4f3466] Submitted process > snp:pileup_variants (3)
Jun-05 11:09:12.042 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:12.042 [Task submitter] INFO  nextflow.Session - [c1/ff5694] Submitted process > snp:pileup_variants (8)
Jun-05 11:09:12.143 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:12.143 [Task submitter] INFO  nextflow.Session - [d2/9b524e] Submitted process > snp:pileup_variants (7)
Jun-05 11:09:12.245 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:12.331 [Task submitter] INFO  nextflow.Session - [d8/f7801c] Submitted process > snp:pileup_variants (1)
Jun-05 11:09:12.343 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:12.431 [Task submitter] INFO  nextflow.Session - [91/6fad08] Submitted process > snp:pileup_variants (13)
Jun-05 11:09:12.444 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:12.532 [Task submitter] INFO  nextflow.Session - [53/d9cea7] Submitted process > snp:pileup_variants (6)
Jun-05 11:09:12.636 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-05 11:09:12.637 [Task submitter] INFO  nextflow.Session - [8f/44913f] Submitted process > snp:pileup_variants (12)
ymcki commented 1 year ago

Instead of running everything in one go, I also tried to run the five tasks individually and see which one crashes. For example, if I run only snp, then

./nextflow run wf-human-variation -w output/workspace -profile singularity --snp --mapula --bam demo_data/demo.bam --bed demo_data/demo.bed --ref demo_data/demo.fasta --basecaller_cfg 'dna_r10.4.1_e8.2_400bps_hac@v3.5.2' --remora_cfg 'dna_r10.4.1_e8.2_400bps_hac@v3.5.2_5mCG@v2' --sample_name MY_SAMPLE --outdir output

I noticed that the same crash happened with --snv. --str but --sv, --cnv, --methyl can finish without errors.

Also, while my system has 2TB RAM, the admin configured zero swap space. Can this be the cause of the crash?

ymcki commented 1 year ago

My admin told me the machine only allocates 4 cores and 8GB RAM to users. I suppose that's why it crashes for runs that require more than 8GB RAM. Sorry for submitting a non-issue.