Closed bioinfo-dirty-jobs closed 7 years ago
Looks like jellyfish isn't installed in the CIPHER img. I fixed this and pushed a commit, you can rebuild the cipher.img file by re-downloading cipher and recreating the singularity image. What is happening here is that CIPHER is trying to calculate the effective genome size since you didn't specify one but for some reason jellyfish didnt get installed.
For now, to overcome this issue without recreating the cipher.img container try the following:
nextflow run -qs 1 cipher.nf -with-singularity cipher.img --mode chip --config example_data/config.txt --fa /mnt/mpwork/Homo_sapiens.GRCh38.rel79.cdna.all.fa --gtf /mnt/mpwork/Homo_sapiens.GRCh38.79.gtf --lib s --readLen 75 --macs_g hs --epic_gn hg38
Here all I did was specify that you are working with hg38 data and gave that information to macs2 and epic for peak calling so that you don't have to automatically calculate it.
This is the output:
ep-13 10:12:18.861 [main] DEBUG nextflow.cli.Launcher - $> /usr/local/bin/nextflow run -qs 1 /opt/cipher-workflow-platform-master/cipher.nf -with-singularity /opt/cipher-workflow-platform-master/cipher.img --mode chip --config /opt/ciphe
r-workflow-platform-master/example_data/config.txt --fa /mnt/mpwork/Homo_sapiens.GRCh38.rel79.cdna.all.fa --gtf /mnt/mpwork/Homo_sapiens.GRCh38.79.gtf --lib s --readLen 75 --macs_g hs --epic_gn hg38
Sep-13 10:12:18.937 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 0.25.7
Sep-13 10:12:18.957 [main] INFO nextflow.cli.CmdRun - Launching `/opt/cipher-workflow-platform-master/cipher.nf` [mad_cajal] - revision: 553a48b852
Sep-13 10:12:18.964 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /opt/cipher-workflow-platform-master/nextflow.config
Sep-13 10:12:18.966 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /opt/cipher-workflow-platform-master/nextflow.config
Sep-13 10:12:19.136 [main] DEBUG nextflow.config.ConfigBuilder - Setting config profile: 'standard'
Sep-13 10:12:19.180 [main] DEBUG nextflow.config.ConfigBuilder - Enabling execution in Singularity container as requested by cli option `-with-singularity null`
Sep-13 10:12:19.200 [main] DEBUG nextflow.Session - Session uuid: 0cf0d174-fdc9-41d2-9f61-1dabd3b36755
Sep-13 10:12:19.201 [main] DEBUG nextflow.Session - Run name: mad_cajal
Sep-13 10:12:19.201 [main] DEBUG nextflow.Session - Executor pool size: 8
Sep-13 10:12:19.210 [main] DEBUG nextflow.cli.CmdRun -
Version: 0.25.7 build 4531
Modified: 06-09-2017 17:25 UTC (19:25 CEST)
System: Linux 3.10.0-327.36.3.el7.x86_64
Runtime: Groovy 2.4.11 on OpenJDK 64-Bit Server VM 1.8.0_111-b15
Encoding: UTF-8 (UTF-8)
Process: 5263@localhost [127.0.0.1]
CPUs: 8 - Mem: 31.1 GB (23.2 GB) - Swap: 2 GB (2 GB)
Sep-13 10:12:19.242 [main] DEBUG nextflow.Session - Work-dir: /mnt/galaxy/TEMP/work [nfs]
Sep-13 10:12:19.520 [main] DEBUG nextflow.Session - Session start invoked
Sep-13 10:12:19.522 [main] DEBUG nextflow.processor.TaskDispatcher - Dispatcher > start
Sep-13 10:12:19.522 [main] DEBUG nextflow.script.ScriptRunner - > Script parsing
Sep-13 10:12:20.250 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Sep-13 10:12:20.270 [main] INFO nextflow.Nextflow -
Sep-13 10:12:20.271 [main] INFO nextflow.Nextflow - RUN INFORMATION
Sep-13 10:12:20.271 [main] INFO nextflow.Nextflow - ===============================================
Sep-13 10:12:20.271 [main] INFO nextflow.Nextflow -
Sep-13 10:12:20.271 [main] INFO nextflow.Nextflow - mode: chip
Sep-13 10:12:20.271 [main] INFO nextflow.Nextflow - config: /opt/cipher-workflow-platform-master/example_data/config.txt
Sep-13 10:12:20.271 [main] INFO nextflow.Nextflow - fasta: /mnt/mpwork/Homo_sapiens.GRCh38.rel79.cdna.all.fa
Sep-13 10:12:20.271 [main] INFO nextflow.Nextflow - gtf: /mnt/mpwork/Homo_sapiens.GRCh38.79.gtf
Sep-13 10:12:20.271 [main] INFO nextflow.Nextflow - library: s
Sep-13 10:12:20.271 [main] INFO nextflow.Nextflow - read-length: 75
Sep-13 10:12:20.271 [main] INFO nextflow.Nextflow -
Sep-13 10:12:20.271 [main] INFO nextflow.Nextflow - subsampled: false
Sep-13 10:12:20.271 [main] INFO nextflow.Nextflow -
Sep-13 10:12:20.271 [main] INFO nextflow.Nextflow - clumpify: true
Sep-13 10:12:20.271 [main] INFO nextflow.Nextflow - bbduk: true
Sep-13 10:12:20.272 [main] INFO nextflow.Nextflow - fastqc: true
Sep-13 10:12:20.272 [main] INFO nextflow.Nextflow - multiqc: true
Sep-13 10:12:20.272 [main] INFO nextflow.Nextflow - mapping: true
Sep-13 10:12:20.272 [main] INFO nextflow.Nextflow - bamcoverage: true
Sep-13 10:12:20.272 [main] INFO nextflow.Nextflow - downstream_analysis true
Sep-13 10:12:20.272 [main] INFO nextflow.Nextflow -
Sep-13 10:12:20.272 [main] INFO nextflow.Nextflow - threads: 1
Sep-13 10:12:20.272 [main] INFO nextflow.Nextflow -
Sep-13 10:12:20.272 [main] INFO nextflow.Nextflow - aligner: bbmap
Sep-13 10:12:20.272 [main] INFO nextflow.Nextflow - ===============================================
Sep-13 10:12:20.272 [main] INFO nextflow.Nextflow -
Sep-13 10:12:20.559 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null
Sep-13 10:12:20.560 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
Sep-13 10:12:20.565 [main] DEBUG nextflow.executor.Executor - Initializing executor: local
Sep-13 10:12:20.567 [main] INFO nextflow.executor.Executor - [warm up] executor > local
Sep-13 10:12:20.572 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=8; memory=31.1 GB; capacity=1; pollInterval=100ms; dumpInterval=5m
Sep-13 10:12:20.576 [main] DEBUG nextflow.processor.TaskDispatcher - Starting monitor: LocalPollingMonitor
Sep-13 10:12:20.577 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local)
Sep-13 10:12:20.580 [main] DEBUG nextflow.executor.Executor - Invoke register for executor: local
Sep-13 10:12:20.613 [main] DEBUG nextflow.Session - >>> barrier register (process: fetch_chrom_sizes)
Sep-13 10:12:20.621 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > fetch_chrom_sizes -- maxForks: 8
Sep-13 10:12:20.661 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null
Sep-13 10:12:20.661 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
Sep-13 10:12:20.661 [main] DEBUG nextflow.executor.Executor - Initializing executor: local
Sep-13 10:12:20.662 [main] DEBUG nextflow.Session - >>> barrier register (process: create_mapping_index)
Sep-13 10:12:20.667 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > create_mapping_index -- maxForks: 8
Sep-13 10:12:20.733 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null
Sep-13 10:12:20.734 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
Sep-13 10:12:20.734 [main] DEBUG nextflow.executor.Executor - Initializing executor: local
Sep-13 10:12:20.735 [main] DEBUG nextflow.Session - >>> barrier register (process: clumpify)
Sep-13 10:12:20.742 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > clumpify -- maxForks: 8
Sep-13 10:12:20.750 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null
Sep-13 10:12:20.750 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
Sep-13 10:12:20.750 [main] DEBUG nextflow.executor.Executor - Initializing executor: local
Sep-13 10:12:20.750 [main] DEBUG nextflow.Session - >>> barrier register (process: bbduk)
Sep-13 10:12:20.757 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > bbduk -- maxForks: 8
Sep-13 10:12:20.762 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null
Sep-13 10:12:20.762 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
Sep-13 10:12:20.762 [main] DEBUG nextflow.executor.Executor - Initializing executor: local
Sep-13 10:12:20.762 [main] DEBUG nextflow.Session - >>> barrier register (process: pre_fastqc)
Sep-13 10:12:20.767 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > pre_fastqc -- maxForks: 8
Sep-13 10:12:20.771 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null
Sep-13 10:12:20.771 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
Sep-13 10:12:20.771 [main] DEBUG nextflow.executor.Executor - Initializing executor: local
Sep-13 10:12:20.772 [main] DEBUG nextflow.Session - >>> barrier register (process: post_fastqc)
Sep-13 10:12:20.777 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > post_fastqc -- maxForks: 8
Sep-13 10:12:20.784 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null
Sep-13 10:12:20.784 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
Sep-13 10:12:20.785 [main] DEBUG nextflow.executor.Executor - Initializing executor: local
Sep-13 10:12:20.785 [main] DEBUG nextflow.Session - >>> barrier register (process: mapping_bbmap)
Sep-13 10:12:20.789 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > mapping_bbmap -- maxForks: 8
Sep-13 10:12:20.811 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null
Sep-13 10:12:20.811 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
Sep-13 10:12:20.811 [main] DEBUG nextflow.executor.Executor - Initializing executor: local
Sep-13 10:12:20.812 [main] DEBUG nextflow.Session - >>> barrier register (process: merge_bams)
Sep-13 10:12:20.818 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > merge_bams -- maxForks: 8
Sep-13 10:12:20.835 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null
Sep-13 10:12:20.835 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
Sep-13 10:12:20.835 [main] DEBUG nextflow.executor.Executor - Initializing executor: local
Sep-13 10:12:20.836 [main] DEBUG nextflow.Session - >>> barrier register (process: bamCoverage)
Sep-13 10:12:20.839 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > bamCoverage -- maxForks: 8
Sep-13 10:12:20.872 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null
Sep-13 10:12:20.872 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
Sep-13 10:12:20.872 [main] DEBUG nextflow.executor.Executor - Initializing executor: local
Sep-13 10:12:20.873 [main] DEBUG nextflow.Session - >>> barrier register (process: multiqc)
Sep-13 10:12:20.876 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > multiqc -- maxForks: 8
Sep-13 10:12:20.879 [Task submitter] INFO nextflow.Session - [65/2ff53e] Submitted process > create_mapping_index
Sep-13 10:12:20.881 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null
Sep-13 10:12:20.882 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
Sep-13 10:12:20.882 [main] DEBUG nextflow.executor.Executor - Initializing executor: local
Sep-13 10:12:20.882 [main] DEBUG nextflow.Session - >>> barrier register (process: estimate_fragment_size)
Sep-13 10:12:20.888 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > estimate_fragment_size -- maxForks: 8
Sep-13 10:14:49.920 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 2; name: create_mapping_index; status: COMPLETED; exit: 0; error: -; workDir: /mnt/galaxy/TEMP/work/65/2ff53e7ad20189fe60f3df848100
4d]
Sep-13 10:14:49.922 [Task submitter] INFO nextflow.Session - [6d/2ff9d2] Submitted process > clumpify (1)
Sep-13 10:14:49.970 [Actor Thread 2] DEBUG nextflow.Session - <<< barrier arrive (process: create_mapping_index)
Sep-13 10:15:19.979 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 3; name: clumpify (1); status: COMPLETED; exit: 0; error: -; workDir: /mnt/galaxy/TEMP/work/6d/2ff9d2e3115e03df843bbc8e87046e]
Sep-13 10:15:21.987 [Task submitter] INFO nextflow.Session - [03/e63e17] Submitted process > clumpify (2)
Sep-13 10:17:05.142 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 4; name: clumpify (2); status: COMPLETED; exit: 0; error: -; workDir: /mnt/galaxy/TEMP/work/03/e63e17f4fb7cb32dffaead5e179b16]
Sep-13 10:17:05.476 [Task submitter] INFO nextflow.Session - [a2/3afd91] Submitted process > fetch_chrom_sizes
Sep-13 10:17:05.769 [Actor Thread 16] DEBUG nextflow.Session - <<< barrier arrive (process: clumpify)
Sep-13 10:17:20.681 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- pending tasks are shown below
~> TaskHandler[id: 1; name: fetch_chrom_sizes; status: RUNNING; exit: -; error: -; workDir: /mnt/galaxy/TEMP/work/a2/3afd91c5c5af3e5bbcbb9c2ba498a5]
Sep-13 10:17:25.279 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: fetch_chrom_sizes; status: COMPLETED; exit: 0; error: -; workDir: /mnt/galaxy/TEMP/work/a2/3afd91c5c5af3e5bbcbb9c2ba498a5]
Sep-13 10:17:25.280 [Task submitter] INFO nextflow.Session - [d6/3dd9b1] Submitted process > pre_fastqc (1)
Sep-13 10:17:25.282 [Task submitter] DEBUG n.processor.TaskPollingMonitor - %% executor local > tasks in the submission queue: 3 -- tasks to be submitted are shown below
~> TaskHandler[id: 6; name: bbduk (1); status: NEW; exit: -; error: -; workDir: /mnt/galaxy/TEMP/work/2e/56684f0708f9eac71a648663556662]
~> TaskHandler[id: 8; name: pre_fastqc (2); status: NEW; exit: -; error: -; workDir: /mnt/galaxy/TEMP/work/0f/9325390b0412b6c23b2c0d2b7141cc]
~> TaskHandler[id: 7; name: bbduk (2); status: NEW; exit: -; error: -; workDir: /mnt/galaxy/TEMP/work/8d/f91845dc16833c7963b5d145ab3e99]
Sep-13 10:17:25.282 [Actor Thread 19] DEBUG nextflow.Session - <<< barrier arrive (process: fetch_chrom_sizes)
Sep-13 10:17:25.285 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null
Sep-13 10:17:25.285 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
Sep-13 10:17:25.286 [main] DEBUG nextflow.executor.Executor - Initializing executor: local
Sep-13 10:17:25.286 [main] DEBUG nextflow.Session - >>> barrier register (process: narrow_peak_calling_WI)
Sep-13 10:17:25.290 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > narrow_peak_calling_WI -- maxForks: 8
Sep-13 10:17:25.295 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null
Sep-13 10:17:25.295 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
Sep-13 10:17:25.295 [main] DEBUG nextflow.executor.Executor - Initializing executor: local
Sep-13 10:17:25.296 [main] DEBUG nextflow.Session - >>> barrier register (process: narrow_peak_calling_NI)
Sep-13 10:17:25.300 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > narrow_peak_calling_NI -- maxForks: 8
Sep-13 10:17:25.326 [main] WARN nextflow.script.ScriptBinding - Access to undefined parameter `epic_egs` -- Initialise it to a default value eg. `params.epic_egs = some_value`
Sep-13 10:17:25.329 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null
Sep-13 10:17:25.330 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
Sep-13 10:17:25.330 [main] DEBUG nextflow.executor.Executor - Initializing executor: local
Sep-13 10:17:25.330 [main] DEBUG nextflow.Session - >>> barrier register (process: broad_peak_calling)
Sep-13 10:17:25.334 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > broad_peak_calling -- maxForks: 8
Sep-13 10:17:25.340 [main] DEBUG nextflow.script.ScriptRunner - > Await termination
Sep-13 10:17:25.340 [main] DEBUG nextflow.Session - Session await
Sep-13 10:17:32.379 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 5; name: pre_fastqc (1); status: COMPLETED; exit: 0; error: -; workDir: /mnt/galaxy/TEMP/work/d6/3dd9b102e6d4c3f603ed09b4c3e7a3]
Sep-13 10:17:32.380 [Task submitter] INFO nextflow.Session - [2e/56684f] Submitted process > bbduk (1)
Sep-13 10:17:32.641 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 6; name: bbduk (1); status: COMPLETED; exit: 1; error: -; workDir: /mnt/galaxy/TEMP/work/2e/56684f0708f9eac71a648663556662]
Sep-13 10:17:32.642 [Task submitter] INFO nextflow.Session - [0f/932539] Submitted process > pre_fastqc (2)
Sep-13 10:17:32.661 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'bbduk (1)'
Caused by:
Process `bbduk (1)` terminated with an error exit status (1)
Command executed:
bbduk.sh in=H3K4me3_Rep1.clumped.fq.gz out=H3K4me3_Rep1.trimmed.fq.gz ref=/opt/cipher-workflow-platform-master/adapters/adapters.fa ktrim=r k=23 mink=11 hdist=1 trimq=6 minlength=10 tpe ordered=t stats=H3K4me3_Rep1_bbduk_statsFile.txt
echo 'bbduk.sh in=H3K4me3_Rep1.clumped.fq.gz out=H3K4me3_Rep1.trimmed.fq.gz ref=/opt/cipher-workflow-platform-master/adapters/adapters.fa ktrim=r k=23 mink=11 hdist=1 trimq=6 minlength=10 tpe ordered=t stats=H3K4me3_Rep1_bbduk_statsFile
.txt' > bbduk_parameters_H3K4me3_Rep1.txt
Command exit status:
1
Command output:
(empty)
Command error:
java -Djava.library.path=/opt/anaconda2/opt/bbmap-37.17/jni/ -ea -Xmx5543m -Xms5543m -cp /opt/anaconda2/opt/bbmap-37.17/current/ jgi.BBDukF in=H3K4me3_Rep1.clumped.fq.gz out=H3K4me3_Rep1.trimmed.fq.gz ref=/opt/cipher-workflow-platform-m
aster/adapters/adapters.fa ktrim=r k=23 mink=11 hdist=1 trimq=6 minlength=10 tpe ordered=t stats=H3K4me3_Rep1_bbduk_statsFile.txt
Executing jgi.BBDukF [in=H3K4me3_Rep1.clumped.fq.gz, out=H3K4me3_Rep1.trimmed.fq.gz, ref=/opt/cipher-workflow-platform-master/adapters/adapters.fa, ktrim=r, k=23, mink=11, hdist=1, trimq=6, minlength=10, tpe, ordered=t, stats=H3K4me3_Re
p1_bbduk_statsFile.txt]
BBDuk version 37.17
Set ORDERED to true
maskMiddle was disabled because useShortKmers=true
Exception in thread "main" java.lang.RuntimeException: Can't read file '/opt/cipher-workflow-platform-master/adapters/adapters.fa'
at shared.Tools.testInputFiles(Tools.java:597)
at jgi.BBDukF.<init>(BBDukF.java:761)
at jgi.BBDukF.main(BBDukF.java:69)
Work dir:
/mnt/galaxy/TEMP/work/2e/56684f0708f9eac71a648663556662
Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
Sep-13 10:17:32.663 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process `bbduk (1)` terminated with an error exit status (1)
Sep-13 10:17:32.690 [Task monitor] DEBUG nextflow.Session - The following nodes are still active:
[process] post_fastqc
status=ACTIVE
port 0: (queue) OPEN; channel: -
port 1: (cntrl) OPEN; channel: $
[process] mapping_bbmap
status=ACTIVE
port 0: (queue) OPEN; channel: -
port 1: (value) - ; channel: ref/*
port 2: (cntrl) OPEN; channel: $
[process] merge_bams
status=ACTIVE
port 0: (queue) OPEN; channel: -
port 1: (cntrl) OPEN; channel: $
[process] bamCoverage
status=ACTIVE
port 0: (queue) OPEN; channel: -
port 1: (cntrl) OPEN; channel: $
[process] multiqc
status=ACTIVE
port 0: (value) - ; channel: fastqc/*
port 1: (value) - ; channel: fastqc/*
port 2: (cntrl) OPEN; channel: $
[process] estimate_fragment_size
status=ACTIVE
port 0: (queue) OPEN; channel: -
port 1: (cntrl) OPEN; channel: $
[process] narrow_peak_calling_WI
status=ACTIVE
port 0: (value) - ; channel: chromSizes
port 1: (queue) OPEN; channel: -
port 2: (cntrl) OPEN; channel: $
[process] narrow_peak_calling_NI
status=ACTIVE
port 0: (value) - ; channel: chromSizes
port 1: (queue) OPEN; channel: -
port 2: (cntrl) OPEN; channel: $
[process] broad_peak_calling
status=ACTIVE
port 0: (value) - ; channel: chromSizes
port 1: (queue) OPEN; channel: -
port 2: (cntrl) OPEN; channel: $
Sep-13 10:17:32.696 [main] DEBUG nextflow.Session - Session await > all process finished
Sep-13 10:17:32.696 [main] DEBUG nextflow.Session - Session await > all barriers passed
Sep-13 10:17:32.703 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local)
Sep-13 10:17:32.726 [main] WARN n.processor.TaskPollingMonitor - Killing pending tasks (1)
Sep-13 10:17:32.790 [main] DEBUG nextflow.CacheDB - Closing CacheDB done
Sep-13 10:17:32.806 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye
The adapters.fa file can't be read from the cipher-workflow-platform/adapters directory. Please make sure it exists and that you have the correct permissions to read/write files from wherever you have the cipher folder. This looks like the /opt/ folder.
To check permissions do ls -l
[maurizio@localhost ANALISI_CHIP]$ nextflow run -qs 1 /opt/cipher-workflow-platform/cipher.nf -with-singularity /opt/cipher-workflow-platform/cipher.img --mode chip --config /opt/cipher-workflow-platform/example_data/config.txt --fa /mnt/mpwork/Homo_sapiens.GRCh38.rel79.cdna.all.fa --gtf /mnt/mpwork/Homo_sapiens.GRCh38.79.gtf --lib s --readLen 75
N E X T F L O W ~ version 0.25.7
Launching `/opt/cipher-workflow-platform/cipher.nf` [hungry_feynman] - revision: 553a48b852
RUN INFORMATION
===============================================
mode: chip
config: /opt/cipher-workflow-platform/example_data/config.txt
fasta: /mnt/mpwork/Homo_sapiens.GRCh38.rel79.cdna.all.fa
gtf: /mnt/mpwork/Homo_sapiens.GRCh38.79.gtf
library: s
read-length: 75
subsampled: false
clumpify: true
bbduk: true
fastqc: true
multiqc: true
mapping: true
bamcoverage: true
downstream_analysis true
threads: 1
aligner: bbmap
===============================================
[warm up] executor > local
WARN: Access to undefined parameter `macs_g` -- Initialise it to a default value eg. `params.macs_g = some_value`
WARN: Access to undefined parameter `epic_egs` -- Initialise it to a default value eg. `params.epic_egs = some_value`
[c2/e39f20] Submitted process > clumpify (2)
[cd/7bf344] Submitted process > create_mapping_index
[a6/48a9d4] Submitted process > clumpify (1)
[43/791166] Submitted process > calculate_egs
[45/c3a76f] Submitted process > fetch_chrom_sizes
[4b/702f86] Submitted process > pre_fastqc (1)
WARN: Access to undefined parameter `epic_gn` -- Initialise it to a default value eg. `params.epic_gn = some_value`
[74/31110d] Submitted process > bbduk (1)
[fa/343544] Submitted process > pre_fastqc (2)
ERROR ~ Error executing process > 'bbduk (1)'
Caused by:
Process `bbduk (1)` terminated with an error exit status (1)
Command executed:
bbduk.sh in=Input_Rep1.clumped.fq.gz out=Input_Rep1.trimmed.fq.gz ref=/opt/cipher-workflow-platform/adapters/adapters.fa ktrim=r k=23 mink=11 hdist=1 trimq=6 minlength=10 tpe ordered=t stats=Input_Rep1_bbduk_statsFile.txt
echo 'bbduk.sh in=Input_Rep1.clumped.fq.gz out=Input_Rep1.trimmed.fq.gz ref=/opt/cipher-workflow-platform/adapters/adapters.fa ktrim=r k=23 mink=11 hdist=1 trimq=6 minlength=10 tpe ordered=t stats=Input_Rep1_bbduk_statsFile.txt' > bbduk_parameters_Input_Rep1.txt
Command exit status:
1
Command output:
(empty)
Command error:
java -Djava.library.path=/opt/anaconda2/opt/bbmap-37.52/jni/ -ea -Xmx5161m -Xms5161m -cp /opt/anaconda2/opt/bbmap-37.52/current/ jgi.BBDukF in=Input_Rep1.clumped.fq.gz out=Input_Rep1.trimmed.fq.gz ref=/opt/cipher-workflow-platform/adapters/adapters.fa ktrim=r k=23 mink=11 hdist=1 trimq=6 minlength=10 tpe ordered=t stats=Input_Rep1_bbduk_statsFile.txt
Executing jgi.BBDukF [in=Input_Rep1.clumped.fq.gz, out=Input_Rep1.trimmed.fq.gz, ref=/opt/cipher-workflow-platform/adapters/adapters.fa, ktrim=r, k=23, mink=11, hdist=1, trimq=6, minlength=10, tpe, ordered=t, stats=Input_Rep1_bbduk_statsFile.txt]
BBDuk version 37.52
Set ORDERED to true
Exception in thread "main" java.lang.AssertionError: Can't find reference file /opt/cipher-workflow-platform/adapters/adapters.fa
at jgi.BBDukF.<init>(BBDukF.java:539)
at jgi.BBDukF.main(BBDukF.java:69)
Work dir:
/mnt/mpwork/DATABASE_TUTORIAL/ANALISI_CHIP/work/74/31110de5099733dfac8ee7bc5dd5ee
Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`
-- Check '.nextflow.log' file for details
Workflow completed on: Thu Sep 14 11:32:55 CEST 2017
Execution status: Failed
Workflow Duration: 5m 43s
Submit issues/requests on GitHub or e-mail cag104@ucsd.edu
You still have the same issue:
Can't find reference file /opt/cipher-workflow-platform/adapters/adapters.fa
Check to make sure that the file exists, and if it does check to make sure you have permissions to read the file in the /opt/ directory. You can do that by running ls -l
from the /opt/ directory.
Here the results of the command you suggest:
[maurizio@localhost cipher-workflow-platform]$ ls -l
total 7064816
drwxrwxrwx. 2 maurizio maurizio 4096 13 set 12.06 adapters
drwxrwxrwx. 4 maurizio maurizio 4096 13 set 12.06 bin
-rwxrwxrwx. 1 maurizio maurizio 8388608032 14 set 10.59 cipher.img
-rwxrwxrwx. 1 maurizio maurizio 77969 13 set 12.06 cipher.nf
drwxrwxrwx. 3 maurizio maurizio 4096 13 set 12.06 docs
drwxrwxrwx. 2 maurizio maurizio 4096 14 set 12.10 example_data
-rwxrwxrwx. 1 maurizio maurizio 57 13 set 12.06 nextflow.config
-rwxrwxrwx. 1 maurizio maurizio 4278 13 set 12.06 README.md
-rwxrwxrwx. 1 maurizio maurizio 1015 13 set 12.06 Singularity
[maurizio@localhost cipher-workflow-platform]$ ls -l adapters/
total 16
-rwxrwxrwx. 1 maurizio maurizio 14034 13 set 12.06 adapters.fa
[maurizio@localhost cipher-workflow-platform]$
So if I try to move to another directory and I run inside the cipher-workflow directory I skyp that error. Now I have this error:
mapping_bowtie2 (2); status: COMPLETED; exit: 0; error: -; workDir: /home/maurizio/Desktop/cipher-workflow-platform/work/b6/bd6fd6ead7791068dac27c38de7d28]
Sep-15 13:24:26.279 [Task submitter] INFO nextflow.Session - [b9/7b4dbb] Submitted process > multiqc
Sep-15 13:24:26.284 [Actor Thread 22] DEBUG nextflow.Session - <<< barrier arrive (process: mapping_bowtie2)
Sep-15 13:24:26.286 [Actor Thread 34] DEBUG nextflow.Session - <<< barrier arrive (process: merge_bams)
Sep-15 13:24:26.305 [Actor Thread 30] WARN nextflow.processor.TaskProcessor - Input tuple does not match input set cardinality declared by process `bamCoverage` -- offending value: [Input, Input_Rep1, /home/maurizio/Desktop/cipher-workflow-platform/work/02/9695b9e3738a10060c1ea30f3a7921/Input_Rep1.sorted.mapped.bam, control1, input, /home/maurizio/Desktop/cipher-workflow-platform/work/02/9695b9e3738a10060c1ea30f3a7921/Input_Rep1.sorted.mapped.bam.bai]
Sep-15 13:24:28.380 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 14; name: multiqc; status: COMPLETED; exit: 0; error: -; workDir: /home/maurizio/Desktop/cipher-workflow-platform/work/b9/7b4dbbb84a93313630cf5a53b05b6d]
Sep-15 13:24:28.380 [Task submitter] INFO nextflow.Session - [da/603a76] Submitted process > estimate_fragment_size (1)
Sep-15 13:24:28.384 [Actor Thread 29] DEBUG nextflow.Session - <<< barrier arrive (process: multiqc)
Sep-15 13:24:30.129 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 15; name: estimate_fragment_size (1); status: COMPLETED; exit: 1; error: -; workDir: /home/maurizio/Desktop/cipher-workflow-platform/work/da/603a761268243fda4b84ec65f1efe2]
Sep-15 13:24:30.130 [Task submitter] INFO nextflow.Session - [9d/95c117] Submitted process > bamCoverage (1)
Sep-15 13:24:30.203 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'estimate_fragment_size (1)'
Caused by:
Process `estimate_fragment_size (1)` terminated with an error exit status (1)
Command executed:
Rscript /home/maurizio/Desktop/cipher-workflow-platform/bin/run_spp.R -c=H3K4me3_Rep1.sorted.mapped.bam -rf -out=H3K4me3_Rep1.params.out -savp=H3K4me3_Rep1.pdf -p=1
Command exit status:
1
Command output:
################
ChIP data: H3K4me3_Rep1.sorted.mapped.bam
Control data: NA
strandshift(min): -500
strandshift(step): 5
strandshift(max) 1500
user-defined peak shift NA
exclusion(min): 10
exclusion(max): NaN
num parallel nodes: 1
FDR threshold: 0.01
NumPeaks Threshold: NA
Output Directory: .
narrowPeak output file name: NA
regionPeak output file name: NA
Rdata filename: NA
plot pdf filename: H3K4me3_Rep1.pdf
result filename: H3K4me3_Rep1.params.out
Overwrite files?: TRUE
[1] TRUE
Reading ChIP tagAlign/BAM file H3K4me3_Rep1.sorted.mapped.bam
opened /tmp/RtmpJgXFxO/H3K4me3_Rep1.sorted.mapped.tagAlign12e532aef457
done. read 252636 fragments
Command error:
Loading required package: caTools
Error: protect(): protection stack overflow
Execution halted
Work dir:
/home/maurizio/Desktop/cipher-workflow-platform/work/da/603a761268243fda4b84ec65f1efe2
Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`
Sep-15 13:24:30.206 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process `estimate_fragment_size (1)` terminated with an error exit status (1)
Sep-15 13:24:30.206 [Actor Thread 34] DEBUG nextflow.Session - <<< barrier arrive (process: estimate_fragment_size)
Sep-15 13:24:30.207 [Actor Thread 34] DEBUG nextflow.Session - <<< barrier arrive (process: narrow_peak_calling_WI)
Sep-15 13:24:30.207 [Actor Thread 22] DEBUG nextflow.Session - <<< barrier arrive (process: broad_peak_calling)
Sep-15 13:24:30.209 [Actor Thread 29] DEBUG nextflow.Session - <<< barrier arrive (process: narrow_peak_calling_NI)
Sep-15 13:24:30.231 [main] DEBUG nextflow.Session - Session await > all process finished
Sep-15 13:24:30.231 [main] DEBUG nextflow.Session - Session await > all barriers passed
Sep-15 13:24:30.231 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local)
Sep-15 13:24:30.250 [main] WARN n.processor.TaskPollingMonitor - Killing pending tasks (1)
Sep-15 13:24:30.315 [main] DEBUG nextflow.CacheDB - Closing CacheDB done
So I don't understand now what is this error? seem now the problem is in the catools So do you suppose to run the pipeline only from the folder of workflow? thanks in advance for any help
`ERROR ~ Error executing process > 'estimate_fragment_size (1)'
Caused by:
Process `estimate_fragment_size (1)` terminated with an error exit status (1)
Command executed:
Rscript /opt/cipher-workflow-platform/bin/run_spp.R -c=H3K4me3_Rep1.sorted.mapped.bam -rf -out=H3K4me3_Rep1.params.out -savp=H3K4me3_Rep1.pdf -p=1
Command exit status:
1
Command output:
################
ChIP data: H3K4me3_Rep1.sorted.mapped.bam
Control data: NA
strandshift(min): -500
strandshift(step): 5
strandshift(max) 1500
user-defined peak shift NA
exclusion(min): 10
exclusion(max): NaN
num parallel nodes: 1
FDR threshold: 0.01
NumPeaks Threshold: NA
Output Directory: .
narrowPeak output file name: NA
regionPeak output file name: NA
Rdata filename: NA
plot pdf filename: H3K4me3_Rep1.pdf
result filename: H3K4me3_Rep1.params.out
Overwrite files?: TRUE
[1] TRUE
Reading ChIP tagAlign/BAM file H3K4me3_Rep1.sorted.mapped.bam
opened /tmp/RtmpPXJtN0/H3K4me3_Rep1.sorted.mapped.tagAlign1be91b6fa48f
done. read 252636 fragments
Command error:
Loading required package: caTools
Error: protect(): protection stack overflow
Execution halted
Work dir:
/opt/cipher-workflow-platform/work/86/1f20538d6efcc4ea1680de5627f8c9
Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
-- Check '.nextflow.log' file for details
Workflow completed on: Fri Sep 15 14:21:37 CEST 2017
Execution status: Failed
Workflow Duration: 18m 10s
Submit issues/requests on GitHub or e-mail cag104@ucsd.edu
WARN: Killing pending tasks (1)
`
No, you should be able to run the pipeline from anywhere. You seem to have some sort of weird permission error on your cluster that doesn't let you read the adapters.fa file from the /opt/ folder. I don't understand it since the permissions look fine. But if you can run CIPHER from inside the cipher-workflow-platform folder then the problem is with your cluster permissions and not with CIPHER. I'd talk to whoever runs the cluster and try to nail down whats going on.
This second error is super interesting! I've NEVER gotten it before, but it seems to be a fairy rare error that occurs when running that particular package that has to do with your FASTA file. It may have too many scaffolds according to this GitHub Issue link: https://github.com/kundajelab/phantompeakqualtools/issues/3
This again unfortunately is not due to a problem with CIPHER but rather another tool (specifically the phantompeakqual tool that CIPHER uses to estimate fragment length). I suggest that you remove all the scaffolds that aren't the major ones (keep chr 1-22, X and Y) from your FASTA file and try running that as the fasta file.
Great thanks so much for your help and patience!!
No, thank you! This is great. I take for granted that CIPHER runs fine on our home machines that it's good to see where it fails outside of those labs I can go into and run myself.
I'm sorry it has been such a hassle getting this to run on your own machine though!
What is the genome file and gtf file do you suggest to use fr chip-seq in your pipeline?
I recommend using the iGenomes files.
The UCSC versions.
I nhave run the tutotial with a machine with 38 G RAM. This is the error: